#internals-and-peps | Python | Page 73

boreal umbra Sep 18, 2020, 4:36 AM

#

But python is the place to be for nlp.

#

That being said, the most recent JetBrains survey about what people are doing with python indicated that data science is the most popular use case in professional or academic settings, or something like that.

trail stratus Sep 18, 2020, 4:57 AM

#

hey what do u mean by attributes in a class

#

is it a function defined inside of a class ?

boreal umbra Sep 18, 2020, 4:58 AM

#

@trail stratus class attributes are a thing, but you might be thinking of attributes that instances of a class have

#

An attribute is basically something you can get with the dot operator

trail stratus Sep 18, 2020, 4:58 AM

#

ohh ye i meant instances

#

ye what are those

boreal umbra Sep 18, 2020, 4:59 AM

#

Well, objects are basically ways of grouping related data and functions together

#

So attributes are all of those things

trail stratus Sep 18, 2020, 4:59 AM

#

well what i mean is suppose there is a question

Make a class called Die with the attribute called sides

SO what is attribute here

boreal umbra Sep 18, 2020, 5:01 AM

#

If you made an instance of the Die class called die, it could have a sidesattribute. And you would get to it from die.sides

trail stratus Sep 18, 2020, 5:01 AM

#

ohh i see thx

boreal umbra Sep 18, 2020, 5:01 AM

#

You could also make a Die instance called pie because why not

trail stratus Sep 18, 2020, 5:02 AM

#

ye, was the question of topic

#

for the channel

boreal umbra Sep 18, 2020, 5:02 AM

#

And pie.sides would give you data of the same type as die.sides

#

But it would be specific to pie instead of die.

#

Generally we wouldn't answer questions in this channel but I went with it because you were asking about how the language is designed

#

And the design of the language is part of the topic. Thanks for asking.

trail stratus Sep 18, 2020, 5:03 AM

#

i see ok thx

cyan bobcat Sep 18, 2020, 5:32 AM

#

Can some tell me how to connect wit.ai in nodejs with sink python script using socket connections and using bash scaript

boreal umbra Sep 18, 2020, 5:41 AM

#

@cyan bobcat take a look at #❓｜how-to-get-help, though if you don't get a response you may want to try again during daytime hours for the US and Europe.

#

And Canada, I guess.

cyan bobcat Sep 18, 2020, 5:43 AM

#

Ok I will try

modern frigate Sep 18, 2020, 5:45 AM

#

@cyan bobcat your question isnt really for this channel, but wit.ai has a python library

empty kite Sep 18, 2020, 8:00 AM

#

How do y'all feel about imports of the typing module? I've started to consistently favour absolute imports of modules for some time now, but at least with the typing module, annotations become unwieldy quickly that way. Right now I do something like

from typing import Generator, List, Tuple

which works fine until you hit the line limit and Black bloats it across multiple lines. Is

from typing import *

really the way to go here? Curious to hear your opinions!

round nexus Sep 18, 2020, 8:03 AM

#

whats wrong with multiple line imports?

radiant fulcrum Sep 18, 2020, 8:04 AM

#

you know you can also use () right

#

📎 unknown.png

empty kite Sep 18, 2020, 8:05 AM

#

That's exactly what I want to avoid

radiant fulcrum Sep 18, 2020, 8:05 AM

#

🤷‍♂️ Typically any wild card imports are a bad idea

#

because it can easily lead to random overrides and poluting your name space

#

if you dont want todo it like the above do import typing as t so you can just do t.Dict for example

#

event though the () system really isnt that bad

empty kite Sep 18, 2020, 8:08 AM

#

Okay so here's a hot take:
If you do

from __future__ import annotations

It should also import all types from typing and also become standard behaviour in Python 4.0

radiant fulcrum Sep 18, 2020, 8:09 AM

#

that would generally just be needless name space poluting tbh

#

3.9 brings general types like list etc... being inbuilt

#

so instead of doing typing.List[str] it will become list[str]

empty kite Sep 18, 2020, 8:10 AM

#

That's nice, I didn't know that

#

I think I will go with multiple import statements from typing instead

peak spoke Sep 18, 2020, 8:30 AM

#

Usually looks fine in parentheses with implicit line continuation, but yeah 3.9 should get the number of them down significantly or moved to other modules

sacred tinsel Sep 18, 2020, 8:31 AM

#

I like 't-dot notation' with import typing as t and then t.List, but others hate it

peak spoke Sep 18, 2020, 8:32 AM

#

I prefer to import them directly but have to admit that the import and the diff you get out of it are not that nice for more than a few names

undone hare Sep 18, 2020, 9:18 AM

#

3.9 will be so nice for that

unkempt rock Sep 18, 2020, 12:30 PM

#

Whats the best encryption program in python?

#

maybe library idk

undone hare Sep 18, 2020, 12:30 PM

#

You should take a look at #❓｜how-to-get-help

#

Also, “best” isn’t a thing in programming

#

It is a form of engineering, there will always be tradeoffs

unkempt rock Sep 18, 2020, 12:34 PM

#

I see, thank u for the tip!

#

Is there a lot of encryption ways?

#

like fernet, and?

undone hare Sep 18, 2020, 12:35 PM

#

#❓｜how-to-get-help :>

paper echo Sep 18, 2020, 1:06 PM

#

@unkempt rock also #cybersecurity

plush root Sep 18, 2020, 1:28 PM

#

dont know where is proper channel for this
so I'm writing library for python but I need inspiration, how can I find?

#

I started but can't find other things for this library

grave jolt Sep 18, 2020, 1:29 PM

#

What are you writing a library for?

heady siren Sep 18, 2020, 1:30 PM

#

how does pypy compare to c?

plush root Sep 18, 2020, 1:30 PM

#

so I have some math methods, validation

grave jolt Sep 18, 2020, 1:31 PM

#

@heady siren PyPy is a Python implementation, and C is a separate language... So can't really compare them

heady siren Sep 18, 2020, 1:31 PM

#

yeah

grave jolt Sep 18, 2020, 1:32 PM

#

If you're doing just number-crunching tasks, it can be at least a few times slower in PyPy than in C

plush root Sep 18, 2020, 1:32 PM

#

hmm general purpose, maybe shrink the range?

heady siren Sep 18, 2020, 1:33 PM

#

seems pretty niche given that numpy exists

grave jolt Sep 18, 2020, 1:33 PM

#

Well, generally, the only way to find out is to implement and measure 🙂

#

@plush root a library that does math things and some kind of validation is probably way to broad and unfocused, yes

plush root Sep 18, 2020, 1:42 PM

#

ok so maybe math lib like extended one

teal yacht Sep 18, 2020, 1:45 PM

#

numpy is not a free speed up for any kind of program

undone hare Sep 18, 2020, 1:45 PM

#

Well, doing computations in numpy instead of pure python is a free speed up

teal yacht Sep 18, 2020, 1:45 PM

#

it's not even that uncommon to see people write numpy code that ends up worse than pure python

#

"well if i use list.append here, i can use np.append right ? what's the problem" @undone hare

#

then you end up with a numpy rewrite that is orders of magnitude slower than the pure python counterpart

undone hare Sep 18, 2020, 1:47 PM

#

Sure, but I’m assuming that the code is correctly written lemon_pleased

grave jolt Sep 18, 2020, 2:01 PM

#

Turns out that the fastest way to add two 2-element tuples elementwise is to write raw bytecode with stack manipulations.

#

And numpy was the slowest method

#

so yes

undone hare Sep 18, 2020, 2:55 PM

#

... lol

visual shadow Sep 18, 2020, 2:56 PM

#

The general advice about the volume of Data is also relevant

#

If you just have very few items, the overhead of creating the numpy array itself could take away any potential speed gains you would have gotten.

vagrant roost Sep 18, 2020, 2:59 PM

#

how can i do the part that said get position of specefic number in lists?

📎 Untitled.png

visual shadow Sep 18, 2020, 2:59 PM

#

Wrong room. Try #❓｜how-to-get-help

hearty monolith Sep 18, 2020, 4:56 PM

#

PEP 584 has this peculiar line: "each of which must be a dict (or an instance of a dict subclass)". Why did the author explicitly mention the part in parenthesis?

flat gazelle Sep 18, 2020, 5:09 PM

#

there is a difference between exactly a dict and a dict subclass, though I cannot think of anything that cannot take a dict subclass and takes exactly dict

spark magnet Sep 18, 2020, 5:14 PM

#

@hearty monolith "be a dict" is ambiguous (because English)

undone hare Sep 18, 2020, 5:17 PM

#

!otn a don't-be-a-dict

hearty monolith Sep 18, 2020, 5:18 PM

#

@spark magnet How come "be a dict" is ambiguous? English isn't my first language

undone hare Sep 18, 2020, 5:19 PM

#

(replace the last letter with a k)

spark magnet Sep 18, 2020, 5:19 PM

#

does it mean type(d) is dict or isinstance(d, dict) ?

undone hare Sep 18, 2020, 5:20 PM

#

isinstance() seeing the part in the parenthesis

hearty monolith Sep 18, 2020, 5:20 PM

#

Wait, those aren't the same?

undone hare Sep 18, 2020, 5:21 PM

#

They aren't, type() will not work on a subclass of dict

hearty monolith Sep 18, 2020, 5:21 PM

#

Ohhh

spark magnet Sep 18, 2020, 5:21 PM

#

@undone hare that's what the doc means, i was answering why "be a dict" is ambiguous.

undone hare Sep 18, 2020, 5:21 PM

#

Oooh, right, I misunderstood

#

The fact that it is between parenthesis is probably because it was added by the person who helped write the PEP, as it was originally ambiguous

hearty monolith Sep 18, 2020, 5:24 PM

#

So, if I understand everything correctly, type(d) is dict asks explicitly if whether d is just a regular dict, while isinstance asks if the dict class was involved with d in any way?

undone hare Sep 18, 2020, 5:25 PM

#

Yeah

#

an example is worth a thousand words, right?

spice pecan Sep 18, 2020, 5:26 PM

#

isinstance can return true for subclass instances or classes that implement an instance hook, while type(d) is dict takes the value returned by type and checks if it's exactly the dict class (thus the is operator)

undone hare Sep 18, 2020, 5:27 PM

#

Yes.

grave jolt Sep 18, 2020, 5:27 PM

#

you can edit it, and it will change the output

undone hare Sep 18, 2020, 5:27 PM

#

I know, I made this feature haha

grave jolt Sep 18, 2020, 5:27 PM

#

ah 🙂

#

😄

undone hare Sep 18, 2020, 5:28 PM

#

Dunno where the error is though haha

flat gazelle Sep 18, 2020, 5:29 PM

#

missing opening quote on last line

hearty monolith Sep 18, 2020, 5:29 PM

#

missing a quote on line 4?

undone hare Sep 18, 2020, 5:29 PM

#

Thank you!

#

!e

class MyDict(dict): pass

my_dict = MyDict()
print('type :', type(my_dict) is dict)
print('isinstance :', isinstance(my_dict, dict))

fallen slateBOT Sep 18, 2020, 5:29 PM

#

@undone hare :white_check_mark: Your eval job has completed with return code 0.

001 | type : False
002 | isinstance : True

undone hare Sep 18, 2020, 5:30 PM

#

It is hard to spot errors on the new mobile codeblock tbh

cloud crypt Sep 18, 2020, 5:38 PM

#

type(x) is faster, depending on your use-case, you might want to use it

spark magnet Sep 18, 2020, 5:46 PM

#

why on earth would speed matter?

grave jolt Sep 18, 2020, 6:06 PM

#

P1. To understand something, you need to access its root
P2. X wants to become an evil genius
P3. To become an evil genius, X must understand evil
P4. Premature optimization is the root of all evil
C1. P2, P3 => X must understand evil
C2. P1, P4 => To understand evil, you need to optimize prematurely
C3. C1, C2 => X must optimize prematurely

#

proof

deft pagoda Sep 18, 2020, 6:15 PM

#

won't accept this until you can show premises are consistent

#

please rewrite in ZF + C

peak pollen Sep 18, 2020, 6:47 PM

#

!otn a don't-be-a-dict

fallen slateBOT Sep 18, 2020, 6:47 PM

#

:ok_hand: Added don’t-be-a-dict to the names list.

peak pollen Sep 18, 2020, 6:47 PM

#

Almost missed it @undone hare

#

Excellent choice

last chasm Sep 18, 2020, 7:09 PM

#

    return list(filter(lambda rule: rule[0] == nonTerms, grammar))```

#

can someone explain this return line to me?

polar carbon Sep 18, 2020, 7:11 PM

#

which OpenCV version is compatible with Python 3.7.6?

cloud crypt Sep 18, 2020, 7:16 PM

#

why on earth would speed matter?
@spark magnet I was working on a [de]serialization framework and compared to type, isinstance is hella slow. I used standard primitive types, so it was exactly what I needed

#

well in the end I wrote some rust which gave me huge speedup but don’t mind that py_guido

spice pecan Sep 18, 2020, 7:18 PM

#

I guess it's predictable

#

Type does what, returns obj.__class__? While isinstance has to check if there are hooks present and adjust accordingly, check the mro...

cloud crypt Sep 18, 2020, 7:45 PM

#

obj.__class__ and type(obj) are the same, except when someone defines __class__ in obj kek

#

and yeah something like obj.__class__ in cls.__mro__ should be not bad

spice pecan Sep 18, 2020, 7:48 PM

#

the point is, both an is check and a type call are really cheap and straightforward while isinstance has a lot going on under the hood, so it's not a surprise that the latter is significantly less efficient

red solar Sep 18, 2020, 11:18 PM

#

Aww they got rid of reactions again

swift imp Sep 19, 2020, 12:38 AM

#

lets say we have this scenario

class A(object):
  def method1():
    pass
  def method2():
    pass

class B(A):
  pass

I want to write a descriptor that effectively blocks method2 on B. Like I don't want that method to exist for B because it would do something undesirable. Can I delete it? Write a descriptor? I'd have to do this a few places and would rather not have to redefine the undesired methods just to raise a NotImplementedError.

#

Nvm I could literally do this

#

class NotDefined:
  def __get__(self, obj, type=None):
    raise NotImplemented

class A(object):
  def method1():
    pass
  def method2():
    pass

class B(A):
  method2 = NotDefined()

#

isnt there some kind of name thing for descriptors?

#

to know what its attached to?

boreal umbra Sep 19, 2020, 12:45 AM

#

!otn a don't-be-a-dict
@peak pollen does this mean we can have python-object-oriented-programming?

swift imp Sep 19, 2020, 12:45 AM

#

__set_name__

forest flicker Sep 19, 2020, 12:58 AM

#

is the python virtual machine considered a LLVM? what's the difference between python VM and LLVM?

boreal umbra Sep 19, 2020, 1:10 AM

#

@forest flicker what is a python virtual machine?

grave jolt Sep 19, 2020, 1:10 AM

#

@forest flicker As far as I know, LLVM is a name of a specific toolset https://en.wikipedia.org/wiki/LLVM, not a general concept

teal yacht Sep 19, 2020, 3:02 AM

#

LLVM also pretty much has nothing to do (anymore) with VMs, despite its name

boreal umbra Sep 19, 2020, 3:55 AM

#

@edgy vault I would open a help channel for that quesiton; see #❓｜how-to-get-help

edgy vault Sep 19, 2020, 3:55 AM

#

ow thanks

half wolf Sep 19, 2020, 7:44 AM

#

@forest flicker VM just means "virtual machine", ie a machine that has no actual silicon. LLVM is a VM because it's targeting a bunch of "virtual machines" in the different steps of the compiler. Very different from the python VM which is a runtime thing.

#

And VM as in VirtualBox/VMWare/etc is a totally different thing too, so it's confusing that there are many things that use the same name.

subtle sequoia Sep 19, 2020, 7:57 AM

#

hey guys 🙂 i’m gonna be bored the next few days and wanna try something new. can anyone recommend some things I should try and make in either a GUI or CLI. this is the advanced discussion so i mean can it be something that might take more than the weekend aha. thank you 🙂 (also ping me if you reply)

robust ruin Sep 19, 2020, 8:19 AM

#

Have you tried getting python macros to work within Gnumeric?

wraith burrow Sep 19, 2020, 8:30 AM

#

Hello, I'm new to python but eventually I want to learn AI, I want to know what IDE to use which would be the best for it. A few people told me before but I want to know from others too.

next cliff Sep 19, 2020, 8:32 AM

#

hey anyone?

can anyone help with OpenCV?

#

i am trying to make hand gesture recoginition

teal yacht Sep 19, 2020, 9:19 AM

#

LLVM is a VM because it's targeting a bunch of "virtual machines" in the different steps of the compiler
LLVM is not a VM

#

The name is a relic of the past, there is some IR that's common for all architectures, then using llc you convert that IR to whatever your machine wants

#

LLVM IR is much closer to a low-level C than something like JVM bytecode

flat gazelle Sep 19, 2020, 9:27 AM

#

LLVM is essentially a compiled to machine code programming language designed to be easy to write for compilers, rather than humans

half wolf Sep 19, 2020, 12:17 PM

#

Two problems with that:

Saying "LLVM is not a VM" is nonsense without defining "VM".
Any IR can be seen as the byte code of a virtual machine that can interpret that byte code.

#

@flat gazelle that doesn't sound right at all. LLVM is absolutely not a programming language by any stretch. It's a toolbox to build compilers.

molten spruce Sep 19, 2020, 2:06 PM

#

Brothers, how can i in my voice assistant add control of openhabian??

neat viper Sep 19, 2020, 2:56 PM

#

how can python reverse a generator?

#

does it exhaust it once and then return them one by one from a new generator?

#

reversed(range(10)) works and returns a iterator

grave jolt Sep 19, 2020, 2:59 PM

#

reversed doesn't accept a generator, it accepts a sequence 🙂

#

range(10) is not a generator -- it's a special range object. It's an iterable sequence, and you can do cool stuff with it.

#

!e

r = range(3, 10)
print(r)
print(r[0], r[1], r[3:6])

fallen slateBOT Sep 19, 2020, 3:00 PM

#

@grave jolt :white_check_mark: Your eval job has completed with return code 0.

001 | range(3, 10)
002 | 3 4 range(6, 9)

neat viper Sep 19, 2020, 3:00 PM

#

ahhh

#

thats cool

grave jolt Sep 19, 2020, 3:01 PM

#

So reversed does something like:

def reversed(sequence):
    for i in range(len(sequence) - 1, -1, -1):
        yield sequence[i]

neat viper Sep 19, 2020, 3:01 PM

#

ahh good to know

#

thanks

spice pecan Sep 19, 2020, 3:04 PM

#

There's also a __reversed__ dunder that you can implement if there's a more efficient way of reverse iteration

grave jolt Sep 19, 2020, 3:06 PM

#

ah, didn't know that

#

!e

print(range.__reversed__)

fallen slateBOT Sep 19, 2020, 3:06 PM

#

@grave jolt :white_check_mark: Your eval job has completed with return code 0.

<method '__reversed__' of 'range' objects>

grave jolt Sep 19, 2020, 3:06 PM

#

yeah

gloomy rain Sep 19, 2020, 3:21 PM

#

@molten spruce This is not a help channel. Check out #❓｜how-to-get-help .

molten spruce Sep 19, 2020, 3:22 PM

#

Okay

charred valve Sep 19, 2020, 3:36 PM

#

        try:
             sensor_data: pd.DataFrame = {
                 'all': lambda: pd.DataFrame([...]),
                 'outside': lambda: pd.DataFrame([...]),
                 'useful': lambda: pd.DataFrame([...]),
                 'family': lambda: pd.DataFrame([...]),
                 'no_child': lambda: pd.DataFrame([...]),
                 'column': lambda: spd.DataFrame([...])
             }[sensor_filter]()
        except KeyError as err:
             raise KeyError(
                 f'Invalid sensor filter supplied: {sensor_filter}') from err

Is there a better way to do this? The context is that the user provides one of these strings, if it matches the dict returns the lambda that generates the data. I dont want to allocate memory for all of the branches, so this way it only creates the dataframe code if there is a match

solemn jewel Sep 19, 2020, 4:19 PM

#

hey um
is there a type annotation for a numeric string

#

like there's string

#

and number

#

but what about both

brave badger Sep 19, 2020, 4:23 PM

#

I mean conceptually that probably wouldn't be a type anymore

solemn jewel Sep 19, 2020, 4:27 PM

#

oh yeah

#

valid point

lone prairie Sep 19, 2020, 4:29 PM

#

why not

grave jolt Sep 19, 2020, 4:52 PM

#

Well, that would be some kind of a dependent type, I suppose 🤷

#

Well, you could have Literal['0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12', ...] 😄

#

Or maybe you could have Sequence[Literal['0', '1', '2', '3', '4', '5', '6', '7', '8', '9']]

#

Yeah, that's probably the closest thing ^

red solar Sep 19, 2020, 4:56 PM

#

Fuck yes let’s introduce dependent types to python 🙂

#

That’s more just c++ templates than full blown dependent types tho (I think?)

grave jolt Sep 19, 2020, 4:58 PM

#

import typing as t

N = t.TypeVar("N")

class Zero:
    pass

class Succ(t.Generic[N]):
    pass

Nat = t.Union[Zero, Succ["Nat"]]

😎

red solar Sep 19, 2020, 4:59 PM

#

Flashbacks to the first thing I learned when a friend was explaining something (in I think Agda?) to me

#

Wonder if someone’s implemented something like this in python

grave jolt Sep 19, 2020, 5:01 PM

#

well, actually

#

it works to a certain very limited extent

red solar Sep 19, 2020, 5:03 PM

#

did u remove bounds?

grave jolt Sep 19, 2020, 5:03 PM

#

yeah, it didn't work for the example for some reason if you leave the bound

#

class Vector(t.Generic[N]):
    pass

def cons(x: t.Any, vec: Vector[N]) -> Vector[Succ[N]]:
    ...

def f(v: Vector[Zero]):
    return cons(42, v)

def g(v: Vector[Succ[Zero]]):
    return cons(42, v)

def h(v: Vector[N]):
    return cons(1,cons(22, v))

Pyright infers f's type to be Vector[Zero] -> Vector[Succ[Zero]], g's type to be Vector[Succ[Zero]] -> Vector[Succ[Succ[Zero]]], and h's type to be Vector[N] -> Vector[Succ[Succ[N]]].

trim glade Sep 19, 2020, 5:10 PM

#

anyone know how simplify this boolean? (p∨¬p)∧(q∨¬q)

grave jolt Sep 19, 2020, 5:11 PM

#

@trim glade True 🙂

#

for further questions, this is a discussion channel (see descrption), not help channel; boolean algebra probably belongs to offtopic

trim glade Sep 19, 2020, 5:11 PM

#

so that is a Tautology then right?

grave jolt Sep 19, 2020, 5:11 PM

#

yes

#

a or (not a) is always true

trim glade Sep 19, 2020, 5:12 PM

#

your right! Thanks a ton

spice pecan Sep 19, 2020, 5:28 PM

#

I gotta do it

#

>>> @ExtremeEdgeCase
... class TechnicallyNotAlways(metaclass=Nitpick):
...     def __init__(self):
...             self.state = False
...     def __bool__(self):
...             val = self.state
...             self.state = not self.state
...             return val
...
>>> a = TechnicallyNotAlways()
>>> a or (not a)
False```

grave jolt Sep 19, 2020, 6:05 PM

#

You should inherit from the Nitpick class

spice pecan Sep 19, 2020, 6:10 PM

#

fixed it

grave jolt Sep 19, 2020, 6:11 PM

#

Technically, you didn't inherit from Nitpick, you used it as a metaclass

#

so you didn't fix it

#

technically

spice pecan Sep 19, 2020, 6:11 PM

#

I feel like a metaclass is more appropriate here, because type(TechnicallyNotAlways) is Nitpick

grave jolt Sep 19, 2020, 6:34 PM

#

@spice pecan Technically, not necessarily -- ExtremeEdgeCase can replace it with something else

spice pecan Sep 19, 2020, 6:34 PM

#

Very true

grave jolt Sep 19, 2020, 6:34 PM

#

I win the nitpick battle then 😎

magic python Sep 19, 2020, 6:41 PM

#

I seem to recall someone mentioning that there was an issue with using return self at the end of a class in order to enable method chaining, I'm not sure what the issue actually was though. What's wrong with that?

grave jolt Sep 19, 2020, 6:44 PM

#

Well, in Python mutating methods return None, it's just a convention

#

curiously, in Smalltalk methods return the instance by default instead of nil

magic python Sep 19, 2020, 6:45 PM

#

there's no reason not to then unless it's something a bunch of others are likely to want to use?

grave jolt Sep 19, 2020, 6:45 PM

#

Well, yes

magic python Sep 19, 2020, 6:45 PM

#

or even in that case i guess it's alright if it's consistent

empty kite Sep 19, 2020, 7:06 PM

#

Guido didn't like fluent interfaces afaik

unkempt rock Sep 19, 2020, 7:17 PM

#

hi guys

#

yo Guido is one of my names

slow reef Sep 19, 2020, 7:19 PM

#

can some one help me with this proplem

magic python Sep 19, 2020, 9:38 PM

#

@empty kite ah, "fluent interfaces", is that what they're called then

#

I'm guessing he's not a big fan of sklearn's design in that case 😄

teal yacht Sep 19, 2020, 10:10 PM

#

Two problems with that:

Saying "LLVM is not a VM" is nonsense without defining "VM".

Any IR can be seen as the byte code of a virtual machine that can interpret that byte code.
@half wolf If we use the regular VM sense of "emulating a computer", then LLVM doesn't come with one, in fact, they are adamant that LLVM should not be called a virtual machine
The LLVM Project is a collection of modular and reusable compiler and toolchain technologies. Despite its name, LLVM has little to do with traditional virtual machines. The name "LLVM" itself is not an acronym; it is the full name of the project.
https://llvm.org/

#

And it's safe to assume lakmatiol was referring to the IR in his message

gleaming rover Sep 19, 2020, 10:49 PM

#

but what about both
@solemn jewel what do you mean?

#

like a value that is either an int or a str?

#

or like a str which can have only numeric characters in it?

#

or something else?

solemn jewel Sep 19, 2020, 10:57 PM

#

or like a str which can have only numeric characters in it?
this @gleaming rover

gleaming rover Sep 19, 2020, 10:57 PM

#

this @gleaming rover
@solemn jewel there isn't

#

but you could make it

#

that'd be a new type

solemn jewel Sep 19, 2020, 10:58 PM

#

oh ok then

unkempt rock Sep 20, 2020, 4:13 AM

#

https://www.python.org/dev/peps/pep-0492/#async-lambda-functions

Python.org

PEP 492 -- Coroutines with async and await syntax

The official home of the Python Programming Language

#

is there a reason why async lambdas were never implemented?

#

i saw https://bugs.python.org/issue33447 this issue of someone asking about it, and they seem like they'd be really handy (@ me)

half wolf Sep 20, 2020, 5:36 AM

#

@teal yacht yea I know all that. Doesn't change anything imo.

brave badger Sep 20, 2020, 6:10 AM

#

@unkempt rock The thing about lambdas is that they're atomic in nature

#

They evaluate just a single expression, unlike coroutines which can be fairly elaborate

unkempt rock Sep 20, 2020, 6:15 AM

#

they're always atomic?

#

didnt know that

brave badger Sep 20, 2020, 6:28 AM

#

Well you wouldn't really want to evaluate more than one expression in a lambda, that would border around being un-Pythonic

flat gazelle Sep 20, 2020, 6:33 AM

#

async lambda: await coro can be lambda: coro and is used about the same.

half wolf Sep 20, 2020, 7:05 AM

#

@brave badger doesn't mean atomic in the normal way here I believe. You can still get a context/thread switch in the middle of a lambda.

#

Hmmm.. Or maybe pure is confused.

#

Well you wouldn't really want to evaluate more than one expression in a lambda, that would border around being un-Pythonic
@brave badger

What does that even mean? Lambdas don't syntactically support anything else than one statement. It's not just unpythonic, it's impossible in python to do more than one.

#

But one statement can also encompass the entire universe of possible computation.

brave badger Sep 20, 2020, 7:12 AM

#

I may be confusing terms, yes haha

#

Lambdas don't syntactically support anything else than one statement. It's not just unpythonic, it's impossible in python to do more than one.
I was referring to using various hacks to do more than one thing at once in a lambda
But one statement can also encompass the entire universe of possible computation.
That's true as well, I guess what I really meant was that having async lambdas wouldn't really add any major benefits given that there's no real way to "compose" coroutines aside from the usual async/await syntax.

#

Also I meant atomic is in does one thing and one thing only

half wolf Sep 20, 2020, 7:44 AM

#

I just consider lambdas to be anonymous functions with an implicit return that are a bit crippled.

#

Which is what they are :)

brave badger Sep 20, 2020, 7:50 AM

#

Fair enough

oak saffron Sep 20, 2020, 8:07 AM

#

I think lamda function is like a fast food things. It's a simple way to create some function

#

It's useful when you need to wrap some operation without create any function block

void thorn Sep 20, 2020, 10:51 AM

#

useful when working with DataFrame : Lambda function

brave badger Sep 20, 2020, 10:57 AM

#

Just curious, how often do coroutines get passed as callbacks?

oak saffron Sep 20, 2020, 11:26 AM

#

Just curious, how often do coroutines get passed as callbacks?
@brave badger it depends on the use case, I think

radiant fulcrum Sep 20, 2020, 11:34 AM

#

@brave badger if its a mostly async system, alot. Any of my webserver and framework stuff supports both coroutine and sync function callbacks because of the nature of this sort of stuff

gentle dock Sep 20, 2020, 12:35 PM

#

oh there is advanced room here

magic python Sep 20, 2020, 1:46 PM

#

I was just looking at itertools.groupby and am not sure where to see what's really going on with it. It seems that it's defined here https://github.com/python/cpython/blob/master/Modules/itertoolsmodule.c#L66 in cpython,

In the output of itertools.groupby?? I have:

Init signature: itertools.groupby(iterable, key=None)
Docstring:
make an iterator that returns consecutive keys and groups from the iterable

I'm not sure how to trace what is going on though, for example in the following:

arr = [(1, "A"), (1, "B"), (1, "C"), (2, "D"), ("k", "E"), (3, "F", "Z")]

for k, g in itertools.groupby(arr, lambda x: x[0]):
    print(list(g))

# outputs:

# [(1, 'A'), (1, 'B'), (1, 'C')]
# [(2, 'D')]
# [('k', 'E')]
# [(3, 'F', 'Z')]

How to know what's going on within the itertools function? Here https://github.com/python/cpython/blob/master/Modules/itertoolsmodule.c#L80 it seems that this is the function that's called? Maybe it's obvious if you know C?

grave jolt Sep 20, 2020, 2:11 PM

#

well, it just initializes an itertools.groupby object

visual shadow Sep 20, 2020, 2:48 PM

#

@magic python couple tips in these kinds of cases: 1. as someone also more comfortable with python over C, i sometimes find tracing functionality in pypy easier for understanding things. and 2. always check documentation first. in this case, they actually provide an almost equivalent python implementation of groupby. check here

#

... okay clearly thats not how hyperlinks are created on discord. 😛

magic python Sep 20, 2020, 2:50 PM

#

@visual shadow cheers, i've never looked at pypy, just heard of it - what would be done in this example wrt pypy?

visual shadow Sep 20, 2020, 2:51 PM

#

basically, the implementation of pypy is essentially guaranteed to be python code. so, same thing. their github source code, and you find your relevant function. just, you'll be guaranteed that its written in python, so it will be easier to read.

#

cpython will implement a lot of core functionality in C code. so it gets harder to trace things down correctly.

#

what often happens is that you end up looking in the wrong place without realising it

magic python Sep 20, 2020, 2:52 PM

#

hm, interesting. I will never call pypy code though presumably, it'll be the cython be default?

#

or is pypy sometimes used without me knowing

visual shadow Sep 20, 2020, 2:52 PM

#

oh. uh. just to clarify terms, cython is something else entirely.

#

and the "implementation" is decided at the time of installing. if you dont know your implementation, it's definitely cpython

#

so, you wont have pypy unless you explicitly install pypy

magic python Sep 20, 2020, 2:55 PM

#

i'm just reading through pypy site, don't really understand why there's a need for both pypy and cpython

#

seems like a lot of duplicated work?

visual shadow Sep 20, 2020, 2:55 PM

#

absolutely. it's entirely duplicated...and yet that's where magic can happen

#

it's like you can get from point A to point B in a car. so why have many different types. 😛

#

they end up having their pros and cons. certain implementations make certain things easier. sometimes you just want certain behaviours that one implementation doesnt give. for example, cpython is particularly good at interfacing with C code. pypy however is JIT compiled, so it can be sometimes quite fast for native python code, more so than cpython

magic python Sep 20, 2020, 2:57 PM

#

I mean - was there originally a question as to which approach was best and Cpython "won" or something? I get different ways for different things, I'm not sure if there would be (in the ideal world of a cpython / pypy dev) just one of these though

visual shadow Sep 20, 2020, 2:58 PM

#

well, mind bendingly enough, pypy is written in python. so, it couldnt really exist till python was invented

#

so in this case, it's more like "lets build something first" and C was the defacto language for doing things at the time i suppose. and pypy as an implementation couldnt exist till python was created

#

C offered a lot of inspiration for guido when he was working on python, and you can sort of see it too.

#

as for the other benefits, i imagine it was the usual, C is fast, etc etc kind of deal

magic python Sep 20, 2020, 3:00 PM

#

so if there was a big meeting and everyone was going to use one or the other from this point, assuming either option was equally easy to implement, pypy would be the more sensible choice?

visual shadow Sep 20, 2020, 3:00 PM

#

sadly, the answer is: it depends. 😄

magic python Sep 20, 2020, 3:00 PM

#

damn it

visual shadow Sep 20, 2020, 3:00 PM

#

if pypy was just blanket better, then probably yes. i dont think that's the case

#

specifically libraries related to C and cython struggle on pypy last i heard. say, numpy and so on

magic python Sep 20, 2020, 3:01 PM

#

pandas 😦

visual shadow Sep 20, 2020, 3:01 PM

#

i will admit im not too sure about the current state of pypy, but i think it was falling behind last i saw.

#

so, if such a meeting were to happen right now, cpython would win just by virtue of "its already ahead. dont fix what aint broken" etc. and yet...theres a clear appetite for JIT compilation, it's why pypy is probably the only other implementation of python that people still talk about

deft pagoda Sep 20, 2020, 3:03 PM

#

the start up time of pypy is awful --- and most python scripts are probably small enough

#

I don't think it would be a good de-facto

#

I'd be more interested in rust-python

magic python Sep 20, 2020, 3:04 PM

#

why'd rust be better than C

deft pagoda Sep 20, 2020, 3:05 PM

#

https://github.com/RustPython/RustPython

GitHub

RustPython/RustPython

A Python Interpreter written in Rust. Contribute to RustPython/RustPython development by creating an account on GitHub.

#

it's not better, it's just another implementation

magic python Sep 20, 2020, 3:06 PM

#

for the fun of it? Or are they making a case that it's an improvement on Cpython

half wolf Sep 20, 2020, 3:06 PM

#

Pypy is behind because cpython is always moving. But the C extensions stuff is mostly fixed afaik. But there are also better ways to make C extensions that make everything better for everyone.

plush sentinel Sep 20, 2020, 3:44 PM

#

does anyone know if I can output the audio of pyaudio to the microphone?
i can't found it in stack overflow

unkempt rock Sep 20, 2020, 4:09 PM

#

Hi... I'm programming a NeunoralNetwork and I am already done with the code and the plot... I just don't know how to get predictions for my examples now.
If you are rly fine with them, please help me... I get desperate ☹️

deft pagoda Sep 20, 2020, 4:18 PM

#

this isn't a help channel

radiant fulcrum Sep 20, 2020, 4:22 PM

#

@deft pagoda Rust python has the potential to be really powerful if they're able todo what they want todo by doing the entire interpreter in pure rust rather than using bits of the C code

#

actually they might have done that already but low api coverage

#

either way it would be amazing to see them do it and actually has the chance to make python more viable in redistribution using WASM etc..

quaint haven Sep 20, 2020, 4:45 PM

#

this is a scary channel 🤣

half wolf Sep 20, 2020, 4:56 PM

#

@plush sentinel send sound TO the microphone? Sure. Plug in the microphone in the speaker jack. It's a very bad speaker though.

#

@radiant fulcrum seems improbable you can get significant gains. Some parts maybe and the security would be much easier to guarantee/validate but that's about it.

radiant fulcrum Sep 20, 2020, 5:01 PM

#

@half wolf I think the most interesting bit is the experimental Jit implementation

#

Which has the potential to provide a much better system compared to PyPy making use of compiler abuse, but this is still a long way off

half wolf Sep 20, 2020, 5:14 PM

#

"compiler abuse"? It's literally a JIT. Some of the problems of pypy not delivering the hoped performance improvements are due to things rust can't fix, but cpython extension emulation and JITs not being enough for example. Rust can't fix those things, it can only fix stuff like the warmup time (which is admittedly a big problem!) and security (which isn't a problem with pypy but with cpython).

charred minnow Sep 20, 2020, 6:04 PM

#

I'm getting an error executing python OOP program that i'm currently working on for college. Hoping if anyone can help me out.

grave jolt Sep 20, 2020, 6:07 PM

#

@charred minnow This is not a help channel, see #❓｜how-to-get-help. You can claim your own help channel and ask here -- and don't ask to ask, just ask your question and attach the code that you have.

cloud crypt Sep 20, 2020, 10:13 PM

#

I always dream about python getting to speeds comparable with javascript

radiant fulcrum Sep 20, 2020, 10:14 PM

#

as much a js has some really weird and awful characteristics, you cant complain at the Node runtime

spiral willow Sep 20, 2020, 10:15 PM

#

Is it pretty speedy?

radiant fulcrum Sep 20, 2020, 10:16 PM

#

compared to python yeah

#

Node is a JIT which is partly responsible for it's speed but overall its alot faster than python usually especially for web frameworks and stuff like that

spiral willow Sep 20, 2020, 10:18 PM

#

I'm always confused why CPython hasn't implemented JIT

radiant fulcrum Sep 20, 2020, 10:20 PM

#

i imagine because of the old issue of the interpreter having such a fucking weird set of quirks

spiral willow Sep 20, 2020, 10:20 PM

#

Maybe in Python4

radiant fulcrum Sep 20, 2020, 10:20 PM

#

js doesnt have anywhere near as much control over the runtime as python though

#

which would cause a issue for alot of systems if python did a similar thing that would both fix stuff but also break alot of stuff

sacred yew Sep 20, 2020, 10:21 PM

#

also python has more complexity than js

#

like dunders

#

plus js needs to be fast

#

because web

radiant fulcrum Sep 20, 2020, 10:21 PM

#

it doesnt really

#

most people's js wont be anywhere near the same speed as what it could be

#

js is designed to be sandboxed tho which python is not

sacred yew Sep 20, 2020, 10:22 PM

#

js development gets more funding because web needs to be fast

radiant fulcrum Sep 20, 2020, 10:22 PM

#

which is why you dont have that high level of runtime exposure as python

sacred yew Sep 20, 2020, 10:22 PM

#

runtime exposure?

radiant fulcrum Sep 20, 2020, 10:23 PM

#

you can do all sorts with python's runtime

sacred yew Sep 20, 2020, 10:23 PM

#

ah

radiant fulcrum Sep 20, 2020, 10:23 PM

#

things that you cant even dream of doing in js which alot of systems like sentry to name one use

sacred yew Sep 20, 2020, 10:23 PM

#

you mean like ctypes?

radiant fulcrum Sep 20, 2020, 10:23 PM

#

no i mean like litterally controlling how the garbage collector works, getting the interpreter frames editing raw memory etc...

oak saffron Sep 20, 2020, 10:51 PM

#

How about FastAPI framework? It said that it's the fastest Python web framework, on par to NodeJS and Go

radiant fulcrum Sep 20, 2020, 10:51 PM

#

its no where near the speeds

#

FastAPI is one of the fastests python frameworks because it runs on uvicorn which is the current fastest python webserver

#

even then Uvicorn is a good 3-4x slower than the fastest NodeJS webserver and go's fastest is a decent 5x faster

oak saffron Sep 20, 2020, 10:53 PM

#

Oh so the fastest python framework still no where near the speeds? lemon_sentimental

radiant fulcrum Sep 20, 2020, 10:54 PM

#

not really

#

Ive made a experimental server that uses Go to bind with python frameworks

#

but that still isnt as fast as Node's fastests stuff

#

at best it gives about a 30% performance increase or so

#

so speed is generally the thing you dont go for with python web backends you go fro the development speed not the execution speed

oak saffron Sep 20, 2020, 10:55 PM

#

I see

radiant fulcrum Sep 20, 2020, 10:55 PM

#

which tbf hardwear is often much cheaper than man hours for a company

#

slower framework -> throw more server at it
slower development speed -> cant really do much

#

why altho Rust has some of the fastest servers for the web its not the most common

#

because its very slow to develop compared to python

oak saffron Sep 20, 2020, 10:56 PM

#

Maybe that's the reason why my company at work still using Python for their server even when their client is banks and stock exchange which is really need speed

radiant fulcrum Sep 20, 2020, 10:57 PM

#

btw if you want to have an idea https://www.techempower.com/benchmarks/ is a 3rd party benchmarker which is generally the 'standard' for comparrisons

www.techempower.com

TechEmpower Web Framework Performance Comparison

Performance comparison of a wide spectrum of web application frameworks and platforms using community-contributed test implementations.

oak saffron Sep 20, 2020, 10:57 PM

#

slower framework -> throw more server at it
slower development speed -> cant really do much
@radiant fulcrum wow, really make sense now haha

#

Yes, i got the information about the comparison from that website too> btw if you want to have an idea https://www.techempower.com/benchmarks/ is a 3rd party benchmarker which is generally the 'standard' for comparrisons
@radiant fulcrum

www.techempower.com

TechEmpower Web Framework Performance Comparison

Performance comparison of a wide spectrum of web application frameworks and platforms using community-contributed test implementations.

grave jolt Sep 20, 2020, 11:07 PM

#

slower framework -> throw more server at it
well... that will improve throughput but might not improve latency.

#

For example, if your website calculates (((2 ** 2) ** 2) ** 2) ... ), you can't improve your speed by buying 1000 servers and running a version of Basic implemented in Python on it

gleaming rover Sep 20, 2020, 11:10 PM

#

For example, if your website calculates (((2 ** 2) ** 2) ** 2) ... ), you can't improve your speed by buying 1000 servers and running a version of Basic implemented in Python on it
@grave jolt that sounds quite beautiful

#

BASIC in Python 🤔

#

but anyway, yeah, I think, relatively speaking, most backend tasks are short-running?

deft pagoda Sep 20, 2020, 11:18 PM

#

beazley did something similar -- emulated a superboard in python

#

https://www.youtube.com/watch?v=pB2t6xG2RzE

YouTube

David Beazley

Using Python 3 To Build a Cloud Computing Service for my Superboard II

David Beazley's PyCon'2011 Talk from Atlanta Georgia. A mirror of original content posted at http://blip.tv/pycon-us-videos-2009-2010-2011/pycon-2011-using-python-3-to-build-a-cloud-computing-service-for-my-superboard-ii-4896844

David Beazley teaches intense in-person Python...

▶ Play video

radiant fulcrum Sep 20, 2020, 11:19 PM

#

@grave jolt yeah latency is another issue

#

python's latency is generally higher than others like compiled frameworks but thats to be expected

oak saffron Sep 20, 2020, 11:21 PM

#

But why Python is mostly used on data science which have a lot of heavy-CPU task if it slow?

feral cedar Sep 20, 2020, 11:23 PM

#

you can call compiled C code in modules like numpy

deft pagoda Sep 20, 2020, 11:24 PM

#

also, data scientists aren't developers --- having a language that's easy to use and slow is better than a language that is fast but difficult to write

oak saffron Sep 20, 2020, 11:29 PM

#

Oh ok I understand now

gleaming rover Sep 20, 2020, 11:29 PM

#

But why Python is mostly used on data science which have a lot of heavy-CPU task if it slow?
@oak saffron most of the CPU-heavy stuff is done in C

#

which, tangentially, is why numpy arrays generally shouldn't contain Python objects

#

In [1]: import numpy as np                                                                                 
In [2]: a = np.arange(1000, dtype=int)                                                                     
In [3]: b = np.arange(1000, dtype=object)                                                                  
In [4]: %timeit a.sum()                                                                                   
2.21 µs ± 11.7 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
In [5]: %timeit b.sum()                                                                                   
22.1 µs ± 376 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

operations in C vs in Python

safe hedge Sep 20, 2020, 11:55 PM

#

Django allows you to define custom commands by putting modules in a set location and uses the following code to detect them:

def find_commands(management_dir):
   """
   Given a path to a management directory, return a list of all the command
   names that are available.
   """
   command_dir = os.path.join(management_dir, 'commands')
   return [name for _, name, is_pkg in pkgutil.iter_modules([command_dir])
           if not is_pkg and not name.startswith('_')]

Are there any potential issues with the this method, particularly the use of pkgutil?

gleaming rover Sep 20, 2020, 11:56 PM

#

@safe hedge what kind of issues are you thinking of

safe hedge Sep 20, 2020, 11:58 PM

#

I don't really know tbh. I always worry about the robustness of code that is introspecting source code itself

raven ridge Sep 21, 2020, 12:00 AM

#

That's not inspecting source code, afaics... It's searching for modules on disk

#

That's not introspection, that's plugins

safe hedge Sep 21, 2020, 12:01 AM

#

But your app code could be installed no?

raven ridge Sep 21, 2020, 12:02 AM

#

that's pretty close to what I mentioned earlier about how git finds subcommands, in fact

safe hedge Sep 21, 2020, 12:02 AM

#

That's not introspection, that's plugins
@raven ridge Yeah this was a pure python example I thought of after our discussion of the git plugin system we mentioned before

#

Haha beat me to it

raven ridge Sep 21, 2020, 12:02 AM

#

😄

safe hedge Sep 21, 2020, 12:03 AM

#

Just wanted to see if anyone had any clear cons with this code tbh

#

One issue I considered is that surely it has to recalculate the available commands every time you call the script. Which seems like unnecessary overhead.

raven ridge Sep 21, 2020, 12:06 AM

#

But your app code could be installed no?
@safe hedge I don't follow this - what does whether it's installed or not have to do with anything?

gleaming rover Sep 21, 2020, 12:07 AM

#

One issue I considered is that surely it has to recalculate the available commands every time you call the script. Which seems like unnecessary overhead.
@safe hedge it's a one-time startup cost

#

which is necessary, because every time you run the available commands could change

raven ridge Sep 21, 2020, 12:07 AM

#

I was about to say the same - doesn't seem unnecessary to me.

gleaming rover Sep 21, 2020, 12:07 AM

#

unless you mean "every time you run the function"

safe hedge Sep 21, 2020, 12:08 AM

#

But if you have the script available as my_command then doing my_command help then my_command task1 help then my_command task1 --opts you've calculated it 3 times

gleaming rover Sep 21, 2020, 12:08 AM

#

But if you have the script available as my_command then doing my_command help then my_command task1 help then my_command task1 --opts you've calculated it 3 times
@safe hedge it's a fast operation

#

relatively speaking, so it doesn't really matter.

#

but anyway the available commands could change between invocations.

safe hedge Sep 21, 2020, 12:08 AM

#

For the cost of doing 1 run. Thought there might be a good way to cache/update the available commands every so often

gleaming rover Sep 21, 2020, 12:09 AM

#

For the cost of doing 1 run. Thought there might be a good way to cache/update the available commands every so often
@safe hedge why?

raven ridge Sep 21, 2020, 12:09 AM

#

Running on a Linux system, by any chance?

gleaming rover Sep 21, 2020, 12:09 AM

#

it's too low cost to matter IMO

safe hedge Sep 21, 2020, 12:09 AM

#

Like maybe have a refresh command that would reread the plugin folder and add new commands

gleaming rover Sep 21, 2020, 12:09 AM

#

excessive cognitive overload IMO

safe hedge Sep 21, 2020, 12:10 AM

#

Kind of like how supervisor reload/update are used to pickup new tasks

#

Running on a Linux system, by any chance?
@raven ridge Mac and Linux yeag

raven ridge Sep 21, 2020, 12:10 AM

#

Like maybe have a refresh command that would reread the plugin folder and add new commands
@safe hedge There's things that work that way, too. vim uses something like that for its :help docs - you need to use :helptags to refresh them.

gleaming rover Sep 21, 2020, 12:10 AM

#

like you could do it, but as they say...

raven ridge Sep 21, 2020, 12:10 AM

#

ever used strace?

gleaming rover Sep 21, 2020, 12:10 AM

#

there are only two hard problems: naming things and cache invalidation

raven ridge Sep 21, 2020, 12:10 AM

#

and off by one errors.

safe hedge Sep 21, 2020, 12:11 AM

#

Fair. This is the sort of advice I'm looking for tbh. I don't know what the overheads for this sort of thing are

gleaming rover Sep 21, 2020, 12:11 AM

#

minimal

#

premature optimisation is the root of all evil

#

unless you have 500 installed apps with 50 commands each

#

then okay, I could see that happening.

safe hedge Sep 21, 2020, 12:11 AM

#

And tbh most of the time I expect these commands to be set to run on a cron more than used interactively

raven ridge Sep 21, 2020, 12:11 AM

#

The first rule of optimization: don't. The second rule of optimization: don't, yet.

gleaming rover Sep 21, 2020, 12:11 AM

#

but you probably have like 20 commands in your project, max?

safe hedge Sep 21, 2020, 12:11 AM

#

So the overhead probably is not significantly problematic

gleaming rover Sep 21, 2020, 12:12 AM

#

I mean

#

Django doesn't even cache its own commands

#

and there are parts of Django which have been optimised to degrees I would say are a little...ridiculous

safe hedge Sep 21, 2020, 12:12 AM

#

but you probably have like 20 commands in your project, max?
@gleaming rover Err probably like 6/7 top level ones, then subcommands

gleaming rover Sep 21, 2020, 12:12 AM

#

okay, maybe not "ridiculous", more "impressive"

raven ridge Sep 21, 2020, 12:13 AM

#

strace is a command that shows all of the system calls that your program makes as it's running.

gleaming rover Sep 21, 2020, 12:13 AM

#

def now():
    if settings.USE_TZ:
        # timeit shows that datetime.now(tz=utc) is 24% slower
        return datetime.utcnow().replace(tzinfo=utc)
    else:
        return datetime.now()

#

^ Django

raven ridge Sep 21, 2020, 12:13 AM

#

I think if you were to ever run your interpreter under strace and look at what it shows you, you'd understand just how trivial the handful of extra disk accesses used for finding plugins are.

gleaming rover Sep 21, 2020, 12:13 AM

#

django.utils.timezone

#

and also, one-time expense (within reason) is a lot more tolerable than something slow that runs in a tight loop

safe hedge Sep 21, 2020, 12:15 AM

#

So in other words. If I were to roughly follow the Django management command model that would be a reasonable solution?

gleaming rover Sep 21, 2020, 12:15 AM

#

yes

#

I have lots of rubbish running on startup and Django still takes less than a second to run for me

#

well, maybe not lots, but some

#

in particular I also have a couple of separate plugin systems that kinda work like that

raven ridge Sep 21, 2020, 12:15 AM

#

~>strace -e stat python3 -c 'import sys' 2>&1 | wc -l
171

Just bringing up the interpreter and importing a single module requires hundreds of stat calls alone.

safe hedge Sep 21, 2020, 12:16 AM

#

I'm thinking of doing it slightly differently by not having a whole Command base class setup, but rather just call the main() in any modules in the right path

gleaming rover Sep 21, 2020, 12:16 AM

#

@safe hedge is this project for your own use?

#

(only)

raven ridge Sep 21, 2020, 12:16 AM

#

your 6/7 extra files don't make a dent.

safe hedge Sep 21, 2020, 12:16 AM

#

It's for work. But effectively I am the main user

gleaming rover Sep 21, 2020, 12:16 AM

#

if someone else will read your code

#

it's generally better to follow conventions that have been around for a long time and make reasonable sense, like this.

#

this seems like a pretty clear case of premature optimisation to me

safe hedge Sep 21, 2020, 12:17 AM

#

this seems like a pretty clear case of premature optimisation to me
@gleaming rover Which bit?

gleaming rover Sep 21, 2020, 12:17 AM

#

I mean, we all love a good day spent prematurely optimising something that is already much faster than it needs to be, but 😉

safe hedge Sep 21, 2020, 12:17 AM

#

The whole plugin idea or the idea of caching the commands

gleaming rover Sep 21, 2020, 12:17 AM

#

@gleaming rover Which bit?
@safe hedge the caching idea

safe hedge Sep 21, 2020, 12:17 AM

#

Oh right yeah I'm more than happy to not do it

#

As I said that's the sort of advice I'm looking for

gleaming rover Sep 21, 2020, 12:18 AM

#

and most people who use Django will expect your commands to have the structure that commands mostly have in Django projects

#

that, more than anything, IMO, is a reason to do things that way

safe hedge Sep 21, 2020, 12:18 AM

#

This is not for Django btw

raven ridge Sep 21, 2020, 12:18 AM

#

for work in particular, there's absolutely no value in making something faster than it needs to be. You're spending the company's money to achieve something in excess of the business requirements imposed by the product owners.

gleaming rover Sep 21, 2020, 12:18 AM

#

oh, okay

safe hedge Sep 21, 2020, 12:18 AM

#

Just using their model as a guide for that plugin system basically

gleaming rover Sep 21, 2020, 12:18 AM

#

then at least don't have a redundant subdirectory

#

like /management/commands 😡

#

(Django)

safe hedge Sep 21, 2020, 12:19 AM

#

Because I want to be able to define new commands without needing to update multiple parts of the codebase

gleaming rover Sep 21, 2020, 12:19 AM

#

and also, yeah, I'm not sure about the subclassing model

#

okay so the topic now is more how to model a plugin system?

safe hedge Sep 21, 2020, 12:21 AM

#

So right now my idea was to have the following structure:

pkg
  - cli
   - command1.py
   - command2.py
  - subpkg
   - code_in_command1_and_2
   - more_code_in_command1_and_2
  - subpkg2
   - code_in_all_commands
   - code_in_some_commands

#

Where basically I can just write modules/subpkgs with the code that does stuff and then have a cli subpkg where I can just write commands which can use any of the code in the other subpkgs

#

So really my commands are separated from the base functionality code but still in the same overarching pkg so it's easier to install

gleaming rover Sep 21, 2020, 12:22 AM

#

okay, fair enough

safe hedge Sep 21, 2020, 12:23 AM

#

then set something like cli.__main__.py to be an entrypoint that runs the whole find_commands part

zealous hawk Sep 21, 2020, 12:34 AM

#

Hello friends, I am having trouble understanding the @staticmethod what is the real use case for this? According to geeksforgeeks, it sounds like it is just a way of getting your ide/linter to stop yelling at you for using self in a method, haha. But I would like to actually understand this better.

gleaming rover Sep 21, 2020, 12:35 AM

#

Hello friends, I am having trouble understanding the @staticmethod what is the real use case for this? According to geeksforgeeks, it sounds like it is just a way of getting your ide/linter to stop yelling at you for using self in a method, haha. But I would like to actually understand this better.
@zealous hawk static methods are methods that are conceptually still linked to a class, but don't require access to either instance attributes or class attributes.

raven ridge Sep 21, 2020, 12:36 AM

#

It's a pretty common opinion that @staticmethod doesn't belong in the language and should never be used. Guido himself has said that he regrets adding it, and that it was only created because he didn't fully understand the problem he was trying to solve

zealous hawk Sep 21, 2020, 12:37 AM

#

Yeah, I understand this much. What is the specific reason to use this instead of a classmethod

i.e why should I do this

class MyClass:
    @staticmethod
    def add():
        return 5 + 5

instead of (obv bad):

class MyClass:
    def add(self):
        return 5 + 5

raven ridge Sep 21, 2020, 12:37 AM

#

It would be better to do:

def add():
    return 5+5

gleaming rover Sep 21, 2020, 12:37 AM

#

It's a pretty common opinion that @staticmethod doesn't belong in the language and should never be used. Guido himself has said that he regrets adding it, and that it was only created because he didn't fully understand the problem he was trying to solve
@raven ridge I agree with the first part of your statement, but Guido isn't always right

raven ridge Sep 21, 2020, 12:38 AM

#

I agree with that, too. Just an example of someone high profile who espouses that view.

#

I'm inclined to agree: @classmethod solves a genuine problem, but every use of @staticmethod could be replaced by a free function.

gleaming rover Sep 21, 2020, 12:38 AM

#

okay, actually sometimes I use staticmethod when I don't want to nest functions in a class

raven ridge Sep 21, 2020, 12:39 AM

#

why not a free function?

gleaming rover Sep 21, 2020, 12:39 AM

#

the convenience it brings is that automated refactoring moves them, too

raven ridge Sep 21, 2020, 12:39 AM

#

like moving the class to a new file?

gleaming rover Sep 21, 2020, 12:39 AM

#

yup

raven ridge Sep 21, 2020, 12:39 AM

#

hm. 🤷

gleaming rover Sep 21, 2020, 12:39 AM

#

which is a super minor thing, and it doesn't happen often 🤷‍♂️

#

sometimes I feel like 4 space indentation is a bit much

#

because methods are already indented once, and then functions nested in methods again

#

but I've gotten used to it

safe hedge Sep 21, 2020, 12:40 AM

#

I like using a staticmethod when I want something that could be a free function to be clearly linked to a class

raven ridge Sep 21, 2020, 12:41 AM

#

if it could be a free function, I'd argue that it shouldn't be linked to a class.

#

something that doesn't depend upon the class or any of its state shouldn't be linked to that class.

zealous hawk Sep 21, 2020, 12:41 AM

#

I like using a staticmethod when I want something that could be a free function to be clearly linked to a class
ty, this is what I thought it was for, but figured there must be more functional use

gleaming rover Sep 21, 2020, 12:42 AM

#

what about non-public helper methods?

safe hedge Sep 21, 2020, 12:42 AM

#

I have also used it for some subclassing stuff. Like I have a DB wrapper and I use a staticmethod which is just pass in the base class to allow some modifications to connection objects in subclasses

zealous hawk Sep 21, 2020, 12:42 AM

#

and ty all, this is very helpful

gleaming rover Sep 21, 2020, 12:42 AM

#

say you have a class that is for drawing PDFs and takes in some complex object; you want to write a method that extracts certain attributes from that object

raven ridge Sep 21, 2020, 12:42 AM

#

what about non-public helper methods?
if they don't use self or cls, then they should be global functions in the module that defines the class.

gleaming rover Sep 21, 2020, 12:42 AM

#

conceptually, it is linked to the class

#

and it wouldn't be useful elsewhere

safe hedge Sep 21, 2020, 12:43 AM

#

But that seems unnecessary if it's only ever going to be used by the class

raven ridge Sep 21, 2020, 12:43 AM

#

if it'll only be used by the class, make it a private method in the module by prefixing it with _

gleaming rover Sep 21, 2020, 12:43 AM

#

it's more a namespacing function in that case, I think

safe hedge Sep 21, 2020, 12:44 AM

#

Another DB example actually. I have a DB that uses 'T/F' for true/false (don't ask) so I have a static method on my client class for that DB which takes a dict and changes all the true/false to T/F

gleaming rover Sep 21, 2020, 12:45 AM

#

if it'll only be used by the class, make it a private method in the module by prefixing it with _
@raven ridge if you have multiple classes in the module, but only one class uses that function, I think it should stay in the class

safe hedge Sep 21, 2020, 12:45 AM

#

It could be a free function but tying it to the client class makes it super clear when and why it should be used

#

And since it's mostly used internally it seems easiest

raven ridge Sep 21, 2020, 12:45 AM

#

I disagree. A static method is a free function that you have namespaced below a class. But if it doesn't depend on that class - it uses nothing from self or cls - then it shouldn't be namespaced below that class. It could be more generally applicable.

gleaming rover Sep 21, 2020, 12:46 AM

#

yes, that's the thing

#

when you use @staticmethod you are using classes-as-namespaces instead of classes-as-data-and-function-containers

#

so I can see the weirdness in that

raven ridge Sep 21, 2020, 12:47 AM

#

yes, exactly. in a nutshell, @staticmethod is a way of using a class as a namespace, but it's not particularly Pythonic to use a class as a namespace. In Python, modules are the idiomatic way of namespacing.

#

the idea of "class as a namespace" is a Java-ish idea - it's something that exists in OOP languages that don't have the ability to create free functions.

red solar Sep 21, 2020, 12:49 AM

#

C++?

safe hedge Sep 21, 2020, 12:49 AM

#

If I have a module with two classes and a function that could be a free function but will only conceivably be used by one of the classes I think it just helps

raven ridge Sep 21, 2020, 12:50 AM

#

I disagree, and my opinion appears to match the majority consensus of Python programmers. Googling for "python staticmethod" will find many more people who espouse my viewpoint.

gleaming rover Sep 21, 2020, 12:51 AM

#

yes, I think it's quite clear that most people are against staticmethod

safe hedge Sep 21, 2020, 12:51 AM

#

I mean, I don't necessarily disagree that they "could" be free functions and I'm not saying you "should" use a staticmethod. But I just don't really see the harm

zealous hawk Sep 21, 2020, 12:51 AM

#

I have always thought it was odd and never really understood the purpose for it. It sounds like it is pretty much only a stylistic consideration.

gleaming rover Sep 21, 2020, 12:51 AM

#

I mean, I don't necessarily disagree that they "could" be free functions and I'm not saying you "should" use a staticmethod. But I just don't really see the harm
@safe hedge but this.

red solar Sep 21, 2020, 12:52 AM

#

if you want to have multiple classes that have no state within a module, i think it's useful

raven ridge Sep 21, 2020, 12:52 AM

#

it is pretty much only a stylistic consideration.
Yes. It's a free function that happens to have been namespaced under a class.

spark magnet Sep 21, 2020, 12:52 AM

#

why have a class with no state?

gleaming rover Sep 21, 2020, 12:53 AM

#

why have a class with no state?
@spark magnet marker classes? don't think that's what they meant though

#

honestly, though, all my staticmethod uses could be nested functions

raven ridge Sep 21, 2020, 12:53 AM

#

we're not talking about class with no state, we're talking about method that doesn't use that state.

red solar Sep 21, 2020, 12:53 AM

#

that too

gleaming rover Sep 21, 2020, 12:53 AM

#

but, as discussed above...indentation + convenience of IDE move

#

and I don't actually know anyone who does what I do so 🤷‍♂️

#

incidentally, does anyone use 2 space indent?

#

because I do that in like every other language...

red solar Sep 21, 2020, 12:54 AM

#

i hope not lol - it's not clear enough for me (2 spaces)

gleaming rover Sep 21, 2020, 12:54 AM

#

which is basically JS/Scala

safe hedge Sep 21, 2020, 12:54 AM

#

I guess I can think of one case where it might make sense? If you have an internal method on a class which is a staticmethod, but it needs to behave differently in some subclass surely it is easiest to just namespace it? Rather than having to name each free function like: class_A_helper, subclass_B_helper, subclass_C_helper?

red solar Sep 21, 2020, 12:54 AM

#

i think jason uses 2 space indents in his c++ series (only person i've seen do it)

gleaming rover Sep 21, 2020, 12:55 AM

#

oh, right

#

I use 4 in C++

red solar Sep 21, 2020, 12:55 AM

#

good 🙂

gleaming rover Sep 21, 2020, 12:55 AM

#

but 2 in JS/Scala?? 🥴

raven ridge Sep 21, 2020, 12:55 AM

#

I have used 2, but I've settled on 4 because that's what most other people use, and using what your team expects is important.

zealous hawk Sep 21, 2020, 12:57 AM

#

I just press tab and hope pycharm is configured 🤷

gleaming rover Sep 21, 2020, 12:59 AM

#

I guess I can think of one case where it might make sense? If you have an internal method on a class which is a staticmethod, but it needs to behave differently in some subclass surely it is easiest to just namespace it? Rather than having to name each free function like: class_A_helper, subclass_B_helper, subclass_C_helper?
@safe hedge this is a good point, I think

#

but then again...is there a concrete example?

red solar Sep 21, 2020, 12:59 AM

#

maybe a per class factory function?

safe hedge Sep 21, 2020, 12:59 AM

#

Dunno, but as I say I've used them in some DB stuff where I modify records before entering depending on the DB

raven ridge Sep 21, 2020, 1:00 AM

#

If it needs to behave differently in the subclass, then it shouldn't be a static method - the subclass's implementation might depend on member variables of that class.

safe hedge Sep 21, 2020, 1:00 AM

#

Whether that was the right thing to do is another Q

raven ridge Sep 21, 2020, 1:00 AM

#

And if it's a factory function, it should almost certainly be a classmethod, not a staticmethod.

red solar Sep 21, 2020, 1:00 AM

#

wait whats the difference?

#

i've only ever used staticmethod

raven ridge Sep 21, 2020, 1:00 AM

#

a classmethod gets the class as an argument.

grave jolt Sep 21, 2020, 1:01 AM

#

@safe hedge Like this?

    if db == "postgresql":
        ...
    elif db == "sqlite":
        ...

safe hedge Sep 21, 2020, 1:01 AM

#

Effectively. But I just have classes that I use for DB interaction

grave jolt Sep 21, 2020, 1:02 AM

#

Well, that's one way to do architecture 🙂

#

there's a classmethod-vs-staticmethod example I copy and paste when explaining it
<#ot1-perplexing-regexing message>

raven ridge Sep 21, 2020, 1:04 AM

#

a classmethod is the right thing to use for a factory function, because it gets the class that it was called on as an argument, which allows it to return a new instance of that class.

#

if you were to use a staticmethod instead, then if you subclassed the class containing that staticmethod and called it, you would still get an instance of the parent class, rather than the child class: it wouldn't know that it had been called on the subclass (and should therefore return an instance of the subclass)

red solar Sep 21, 2020, 1:05 AM

#

ahh ok

#

(i've never actually written a factory function in python)

trail stratus Sep 21, 2020, 7:57 AM

#

can this not unpack int()?

        for key, value in scores.values():
            print(f"\n Computer : {key}")
            print(f"\t You :{value}")

#

if this question is not suited for this channel pls tell me so

undone hare Sep 21, 2020, 7:58 AM

#

Sounds like a question you should ask in an help channel lemon_pleased

#

See #❓｜how-to-get-help

trail stratus Sep 21, 2020, 7:58 AM

#

ok

teal yacht Sep 21, 2020, 7:58 AM

#

it isn't suited for this channel, but there is nothing wrong with your code just by looking at it, assuming scores is a dict

trail stratus Sep 21, 2020, 8:07 AM

#

ok thx

unkempt rock Sep 21, 2020, 8:08 AM

#

values() only returns a single value, you should be using items

trail stratus Sep 21, 2020, 8:08 AM

#

i got it thx

teal yacht Sep 21, 2020, 8:13 AM

#

oh woops

trail stratus Sep 21, 2020, 8:18 AM

#

for what

grand ibex Sep 21, 2020, 9:26 AM

#

Hey guys, Is machine learning and the fields of AI overhyped and over saturated? Will it still be the hottest job in the future like 2020-2040? I see everyone taking ML courses, reading books and getting a job. Is ML that easy? Will it still remain the most indemand jobs in the future?

unkempt rock Sep 21, 2020, 9:30 AM

#

@grand ibex Hey guys, Is machine learning and the fields of AI overhyped and over saturated? No it's not
Will it still be the hottest job in the future like 2020-2040? No one can predict the future acuratly but as things are looking now YES it is going to play a big part
I see everyone taking ML courses, reading books and getting a job. Is ML that easy?
No it is not easy to learn in debt but anyone can learn the basic surface of it
Will it still remain the most indemand jobs in the future?
Most likely yes

half wolf Sep 21, 2020, 10:47 AM

#

I disagree with all of those :P

#

Unless something radical happens with explainable ML models, the fundamental problem is that ML is a black box. Lots and lots of business rules need to be able to answer the simple question "why did the system do this?" Might even be a legal problem if you can't.

#

Like this thing: https://www.theverge.com/2020/9/20/21447998/twitter-photo-preview-white-black-faces

teal yacht Sep 21, 2020, 10:54 AM

#

while explainability is a real problem, it isn't entirely blocking for most companies, ML is used in finance, energy, ecology, defense, and many other fields and there's no reason it would stop all of a sudden

#

let alone for companies that use ML for more "mundane" tasks

half wolf Sep 21, 2020, 11:01 AM

#

Sure. But it's also not clear that there's a growth in demand that is going to outstrip supply for all time. Just like with programming it didn't actually turn out to be a career where the value of the programmer was slashed when there were enough of us (there never was), or when everyone could do their programming themselves (never happened).

#

ML is going to be useful for the foreseeable future for sure, but if the talent pool is the same as for regular programmers (which is seems like), it would seem like both ML arborists and regular old programmers can still enjoy great salaries for a long time to come.

#

Depends on what "most in demand" means I guess...

#

In Stockholm programmer is the most common job. That's one way to say "most in demand"... but that's not a very useful metric imo.

deft pagoda Sep 21, 2020, 11:29 AM

#

this is offtopic for this channel in anycase, try #career-advice

#

this isn't a help channel

#

please see #❓｜how-to-get-help

grand ibex Sep 21, 2020, 12:34 PM

#

@unkempt rock thanks for the answer!

covert saddle Sep 21, 2020, 4:45 PM

#

class RandomGenerator:
    """ The base class for all random markdown generators """

    file_name: str
    output_dir: str
    output_size: int

    def __init__(self,
                 file_name=None,
                 output_size=None,
                 output_dir='.'):
        self.file_name = file_name
        self.output_size = output_size
        self.output_dir = output_dir

Here, I want to raise an error if file_name is not provided (or set to None) during class initialization. Which method should I use? I was thinking __getattr__ but I am not really sure.
Or should I use a try..except block in __init__ itself?

wide shuttle Sep 21, 2020, 4:47 PM

#

I'm not quite sure what you mean; what's stopping you from checking if the parameter file_name is assigned to None and then raising an exception?

#

You could also just make it a mandatory argument by not providing a default

covert saddle Sep 21, 2020, 4:48 PM

#

I'm not quite sure what you mean; what's stopping you from checking if the parameter file_name is assigned to None and then raising an exception?
@wide shuttle My question was, where do I check it. In the __init__? And using what method? I was thinking try..except but idk

#

You could also just make it a mandatory argument by not providing a default
@wide shuttle That is a good solution

#

But if I needed to check for some different condition, where would I do that?

dusty dune Sep 21, 2020, 5:35 PM

#

if i have multiple columns with a single letter in one dataframe, and another dataframe has a weight for each letter, how do I multiply the weights?

warm raven Sep 21, 2020, 5:50 PM

#

do you guys play wow? just wondering

wide shuttle Sep 21, 2020, 5:52 PM

#

This channel is specifically meant to discuss Python itself, from a higher-level and more abstract perspective. For WoW, I'd recommend an offtopic channel. (There are three of them just below in the offtopic/general category.)

half wolf Sep 21, 2020, 5:52 PM

#

@covert saddle why are you making the parameter optional then making it required also? That's just confusing.

covert saddle Sep 21, 2020, 5:52 PM

#

@half wolf I was wrong there, so I fixed it

half wolf Sep 21, 2020, 5:53 PM

#

Raising exceptions from the constructor is perfectly fine.

covert saddle Sep 21, 2020, 5:54 PM

#

So __init__ would be okay place to raise exception?

half wolf Sep 21, 2020, 6:19 PM

#

Absolutely. This isn't C++. Python is much more sane :)

radiant fulcrum Sep 21, 2020, 6:22 PM

#

I love that xD

raven ridge Sep 21, 2020, 6:22 PM

#

Python isn't much different than C++ here, actually... It's sane to raise an exception from a constructor in both, and it's not safe to raise an exception from the destructor in either

#

Though Python handles it more sanely if you do.

boreal umbra Sep 21, 2020, 6:28 PM

#

@raven ridge I thought raising an exception in __del__ just gets suppressed

#

though I guess that results in unsafe behavior

#

if your __del__ calls something and you need exceptions it raises to be handled.

#

(even if you're not raising one right in the __del__)

peak spoke Sep 21, 2020, 6:30 PM

#

del is a bit special, have yet to see it used anywhere with all the special cases around it

boreal umbra Sep 21, 2020, 6:30 PM

#

I've never used it

grave jolt Sep 21, 2020, 6:44 PM

#

Maybe it should've been called __on_garbage_collection__?

#

Because __del__ suggests that it's a hook for del somehow

mystic cargo Sep 21, 2020, 6:48 PM

#

yeah, that's what I thought at first

#

but searching it up made it clear that it means __on_garbage_collection__

red solar Sep 21, 2020, 7:22 PM

#

@raven ridge so are my destructors implicitly noexcept?

deft pagoda Sep 21, 2020, 7:30 PM

#

I used __del__ once

#

but rewrote the code later to not need it

raven ridge Sep 21, 2020, 7:50 PM

#

@raven ridge I thought raising an exception in __del__ just gets suppressed
@boreal umbra printed to stderr and suppressed, yes - which is saner than the C++ behavior of just aborting the program.

#

@raven ridge so are my destructors implicitly noexcept?
@red solar https://docs.python.org/3/reference/datamodel.html#object.__del__ - yeah.

Warning

Due to the precarious circumstances under which __del__() methods are invoked, exceptions that occur during their execution are ignored, and a warning is printed to sys.stderr instead

red solar Sep 21, 2020, 7:53 PM

#

i meant more in c++

#

somehow i'm more familiar with implementing del in C, than C++ destructors :/

raven ridge Sep 21, 2020, 8:02 PM

#

In modern C++ destructors default to noexcept, yes.

#

The point I'm making is just that both languages face exactly the same problem and handle it in roughly the same way.

#

C++ is less friendly about it, just terminating your program, but both say it's the user's responsibility to prevent a destructor from throwing.

covert saddle Sep 22, 2020, 2:12 AM

#

How can I have optional dependencies for my package? For example in django, unless you use models.ImageField, you don't need to have PIL installed.
If I write a module using that optional dependency, do I need to write it separately to prevent any ImportError?

sacred yew Sep 22, 2020, 2:25 AM

#

use extras_require @covert saddle

#

example:

extras_require = {
    'voice': ['PyNaCl==1.3.0'],
    'docs': [
        'sphinx==1.8.5',
        'sphinxcontrib_trio==1.1.1',
        'sphinxcontrib-websupport',
    ]
}

covert saddle Sep 22, 2020, 2:26 AM

#

Where? I am using poetry package manager to build and publish my package

sacred yew Sep 22, 2020, 2:26 AM

#

uhh

#

this is an arg to setuptools.setup

#

let me find what the poetry equiv is

covert saddle Sep 22, 2020, 2:27 AM

#

https://github.com/python-poetry/poetry

GitHub

python-poetry/poetry

Python dependency management and packaging made easy. - python-poetry/poetry

sacred yew Sep 22, 2020, 2:27 AM

#

https://python-poetry.org/docs/pyproject/#extras

The pyproject.toml file | Documentation | Poetry - Python dependenc...

Official documentation of Poetry

#

@covert saddle ^ equiv for poetry

covert saddle Sep 22, 2020, 2:28 AM

#

oh okay.

#

If I write a module using that optional dependency, do I need to write it separately to prevent any ImportError?

What about this?

safe hedge Sep 22, 2020, 4:04 AM

#

If you want to make sure it doesn't cause an ImportError then yes you need to make sure you structure it in the correct way

#

For example:

pkg
 - __init__.py
   - subpkg
     - __init__.py
     - module.py
   - subpkg_that_has_optional
     - __init__.py
     - module.py

Now if you were to put import subpkg_that_has_optional or from subpkg_that_has_optional in your top-level __init__.py then any time you try to import that package at all you'll get an ImportError if the optional thing isn't installed

#

@covert saddle

covert saddle Sep 22, 2020, 4:23 AM

#

@safe hedge okay, so i shouldn't be importing subpkg_that_has_optional in my top level __init__.py, right?

half wolf Sep 22, 2020, 4:39 AM

#

@covert saddle you need to put some import statements either inside functions and/or inside try/except ImportError blocks.

safe hedge Sep 22, 2020, 5:27 AM

#

Well it depends right

#

You can just not import them at all in the __init__.py

#

And then the users have to explicitly import those subpackages?

#

And that way the import error should only occur if someone is explicitly trying to import the module that uses optional stuff

#

Or you could do something like:

#

try:
    import module_with_optional
except ImportError:
    pass

in your __init__.py so that the module with optional stuff is made available simply by importing the top-level package if the optional module can be imported

covert saddle Sep 22, 2020, 5:37 AM

#

okay thanks @WelshWizard#1004 @boxed#9332

covert saddle Sep 22, 2020, 7:45 AM

#

Can someone tell me if Test Driven Development (TDD) is actually used in the industry? Is it a good idea to approach a new project using TDD?

brazen jacinth Sep 22, 2020, 7:54 AM

#

yes to both

gloomy rain Sep 22, 2020, 7:58 AM

#

@covert saddle In my career I've used TDD as one tool among many. It makes more sense to use in some contexts than in others. If you can formulate your problem in terms of input and output and you have a clear idea of what you want your code to do, but not necessarily exactly how to do it, I think TDD really shines.

terse orchid Sep 22, 2020, 8:37 AM

#

Hello, can I consult here for best practices for my project? I doesn't consists of only python though.

covert saddle Sep 22, 2020, 8:39 AM

#

How much do I have to worry about backwards compatibility when writing a new package? I am using a lot of f-strings which will not work in python 2.7

#

Do people still use python 2.7?

narrow pasture Sep 22, 2020, 8:40 AM

#

I don’t think so

covert saddle Sep 22, 2020, 8:41 AM

#

So I don't have to worry?

gloomy rain Sep 22, 2020, 8:45 AM

#

Well, we do at my employer.

#

We're unable to upgrade due to being stuck between market constraints and not having a good migration path.

#

But since 2.7 reached eol, you probably shouldn't bother accounting for such cases.

#

It's our problem, not yours.

covert saddle Sep 22, 2020, 8:51 AM

#

hmm okay thanks for the advise

terse orchid Sep 22, 2020, 8:57 AM

#

Okay here I go. I have developed a SaaS product for our company. I'm at the back end and I use python, mongodb and flask. On the front, angular is being used.

Our product consists of a dashboard and couple of other tabs to present the data. It mainly focuses on text processing and making insights from text data. In dashboard, it has many charts, summaries etc.

For now it works. But I'm not sure if I made it with using best practices. Now, whenever a user opens the site (dashbord), calculations are being made for many different charts. I'm saving some of the charts to cache (simply a local file) and re-use it if it haven't requested recently. What is this kind of application's best practice? Which tools and methods I can use for both maintaining and not to use redundant re-calculations? Text processing is sometimes an expensive process. Sorry if here is the wrong place to ask this or the question is too broad, but I don't know anywhere else to ask.

languid dagger Sep 22, 2020, 9:27 AM

#

@covert saddle You should never have to worry about backwards compatibility unless you want to support it. You just have it so that your package is limited to specific versions and they'd need to use the matching version of Python to use your package. f-strings came in at 3.6 so you can support that and up. If you want to support <=3.5 you'd have to use .format

covert saddle Sep 22, 2020, 9:38 AM

#

f-strings are much better than .format imo, but I do agree to what you say @languid dagger

brave badger Sep 22, 2020, 9:43 AM

#

iirc I'm starting to see newer versions of open-source libs dropping support for Python 3.5

half wolf Sep 22, 2020, 9:48 AM

#

@covert saddle also think about the passage of time. Say you write a package today: first there are some pretty big odds no one will care at all (yea that sucks) so then don't care about old python versions! Then if people do it will take a long time for the word to get around, again meaning by the time your lib is actually popular the definition of "old" will be different.

#

The smaller the lib, the more aggressive you should be in dropping old python versions.

#

"Small" is in popular/widely used.

covert saddle Sep 22, 2020, 10:30 AM

#

okay thanks @half wolf that was really helpful advise!

stone mauve Sep 22, 2020, 10:34 AM

#

I made my first package today

covert saddle Sep 22, 2020, 10:43 AM

#

Sorry for asking questions in installments, but I am asking them as they come up in mind.

the more aggressive you should be in dropping old python versions.
Does this mean I don't need to use tools like tox at all? Will pytest and flake8 suffice in testing my package? @half wolf

half wolf Sep 22, 2020, 10:55 AM

#

@covert saddle I would think so. tox can be useful for running inside travis or whatever too probably though. And you might want to support multiple versions at some point.

covert saddle Sep 22, 2020, 10:56 AM

#

okay and thanks again

unkempt rock Sep 22, 2020, 11:34 AM

#

ok

safe hedge Sep 22, 2020, 11:41 PM

#

f-strings are much better than .format imo, but I do agree to what you say @languid dagger
@covert saddle I seem to find myself repeating this a lot lately. f-strings are NOT a 1-to-1 replacement for format even in 3.6+

grave jolt Sep 22, 2020, 11:48 PM

#

Well, of course; there are cases where 1) format is more concise (like when accessing a lot of self's attributes; 2) you need to pass a template

spark magnet Sep 22, 2020, 11:49 PM

#

@safe hedge did you have other cases?

grave jolt Sep 22, 2020, 11:52 PM

#

when the expression includes a backslash, maybe?

#

but in that case it's probably better to extract the expression into a variable anyway

safe hedge Sep 22, 2020, 11:54 PM

#

3 is particularly annoying imo

#

I regularly want to do something like f"Some content \n{'\n'.join(somelist)} somemorecontent {variable}" etc

grave jolt Sep 22, 2020, 11:55 PM

#

chr(10).join(somelist) 😄

safe hedge Sep 22, 2020, 11:56 PM

#

I just find that too many people seem to think .format is totally dead

covert saddle Sep 23, 2020, 12:03 AM

#

I certainly did, since I have never seem to use them after I discovered f strings

safe hedge Sep 23, 2020, 12:04 AM

#

In general it might even be true. But at some point someone may come across a problem with f-strings and not understand they need to drop back to .format. I think it's important to make the distinction between them.

gleaming rover Sep 23, 2020, 12:05 AM

#

"There should be one - and only one - obvious way to do it."

#

🤔

safe hedge Sep 23, 2020, 12:07 AM

#

Ha

gleaming rover Sep 23, 2020, 12:07 AM

#

that part of the Zen is pretty much in ruins

spark magnet Sep 23, 2020, 12:07 AM

#

@safe hedge i think the people who think format is dead are the people who only ever used format in ways that can now become f-strings (and should).

safe hedge Sep 23, 2020, 12:08 AM

#

Let's not forget that you should still use % formatting for logging too.

#

String formatting: not even once

gleaming rover Sep 23, 2020, 12:08 AM

#

Let's not forget that you should still use % formatting for logging too.
@safe hedge I am split on this

#

on one hand, yes, efficiency, etc.

#

on the other, readability counts, and as long as you're not logging a lot in a tight loop...

teal yacht Sep 23, 2020, 12:08 AM

#

you don't have to tho, you can change the style

gleaming rover Sep 23, 2020, 12:08 AM

#

but you know what we really need?

safe hedge Sep 23, 2020, 12:09 AM

#

I quite like it tbh because it makes it really easy for me to spot which strings are for logging at a glance

gleaming rover Sep 23, 2020, 12:09 AM

#

call-by-name

#

🙂

fathom tusk Sep 23, 2020, 12:09 AM

#

can anyone help me with this plz

safe hedge Sep 23, 2020, 12:09 AM

#

you don't have to tho, you can change the style
@teal yacht You can change the % to something else, but it's still old-style formatting

fathom tusk Sep 23, 2020, 12:09 AM

#

📎 image0.png

safe hedge Sep 23, 2020, 12:09 AM

#

You don't do .format or f-strings in logging for efficiency

gleaming rover Sep 23, 2020, 12:09 AM

#

@fathom tusk try #❓｜how-to-get-help

fathom tusk Sep 23, 2020, 12:09 AM

#

ok

#

but if u get an answer lmk

spark magnet Sep 23, 2020, 12:10 AM

#

@gleaming rover you are really dipping deep into cs history with "call by name"

gleaming rover Sep 23, 2020, 12:10 AM

#

but if u get an answer lmk
@fathom tusk I know how to solve it, it's a p basic question

#

but it is also off-topic for this channel

teal yacht Sep 23, 2020, 12:10 AM

#

the only "old-style" thing is the %, not how the arguments are evaluated and added to the string

#

if you use {} it's no longer "old-style"

gleaming rover Sep 23, 2020, 12:10 AM

#

@gleaming rover you are really dipping deep into cs history with "call by name"
@spark magnet well...

fathom tusk Sep 23, 2020, 12:10 AM

#

dm me the answers? @gleaming rover would be greatly appreciated =]

gleaming rover Sep 23, 2020, 12:10 AM

#

I had to Google it to make sure I had the name right

spark magnet Sep 23, 2020, 12:11 AM

#

algol, right?

gleaming rover Sep 23, 2020, 12:11 AM

#

I only know about it because of Scala, which I picked up last year

#

nope

spark magnet Sep 23, 2020, 12:11 AM

#

scala does that?

gleaming rover Sep 23, 2020, 12:11 AM

#

I wasn't even close to being born when ALGOL was invented

#

in fact, I don't even think my parents were born

spark magnet Sep 23, 2020, 12:11 AM

#

you could still know about it 🙂

gleaming rover Sep 23, 2020, 12:11 AM

#

dm me the answers? @gleaming rover would be greatly appreciated =]
@fathom tusk that looks like homework and is something you could probably solve yourself. do a little Googling.

teal yacht Sep 23, 2020, 12:12 AM

#

why would you want that tho @gleaming rover ?

gleaming rover Sep 23, 2020, 12:12 AM

#

scala does that?
@spark magnet what I understand by "call-by-name", in the way Scala uses it, is an argument passing strategy where unevaluated expressions are passed

#

and they are only evaluated as needed

teal yacht Sep 23, 2020, 12:12 AM

#

hacking macros together into python ?

#

no thanks

spark magnet Sep 23, 2020, 12:12 AM

#

right, like macros

gleaming rover Sep 23, 2020, 12:12 AM

#

why would you want that tho @gleaming rover ?
@teal yacht it was facetious

#

to solve the problem of logging inefficiency

spark magnet Sep 23, 2020, 12:12 AM

#

people have macro things for python

gleaming rover Sep 23, 2020, 12:12 AM

#

only evaluate the expressions if the logging call is actually needed

teal yacht Sep 23, 2020, 12:13 AM

#

it's kinda impossible to know if a logging call is needed tho

#

unless you meant the flushing part i guess

spark magnet Sep 23, 2020, 12:13 AM

#

a thing people often overlook when proposing new features for python is that each file is compiled separately, with no information from the imported files.

safe hedge Sep 23, 2020, 12:14 AM

#

@teal yacht Ok. Let's call it a 3rd style then.

gleaming rover Sep 23, 2020, 12:14 AM

#

it's kinda impossible to know if a logging call is needed tho
@teal yacht what do you mean

teal yacht Sep 23, 2020, 12:14 AM

#

how do you decide whether or not a log will be needed ?

#

we delay string interpolating in logging libraries because IO is expensive, to do it in batches, and strings are more expensive than references to objects

spark magnet Sep 23, 2020, 12:15 AM

#

i don't understand that last point

gleaming rover Sep 23, 2020, 12:15 AM

#

how do you decide whether or not a log will be needed ?
@teal yacht logging level?

teal yacht Sep 23, 2020, 12:16 AM

#

ah yeah i guess i forgot about that point

gleaming rover Sep 23, 2020, 12:17 AM

#

yeah so like logging is built the way it is to delay string interpolation, right

raven ridge Sep 23, 2020, 12:17 AM

#

it's kinda impossible to know if a logging call is needed tho
@teal yacht it's to only format the message if the log record passes the filters.

gleaming rover Sep 23, 2020, 12:17 AM

#

but the arguments are still evaluated

teal yacht Sep 23, 2020, 12:17 AM

#

@spark magnet we don't do log("%s" % obj), we do log("%s", obj)

raven ridge Sep 23, 2020, 12:17 AM

#

Evaluated but not interpolated

gleaming rover Sep 23, 2020, 12:17 AM

#

so call-by-name (which I am not seriously suggesting, just for this), would also delay the evaluation of arguments

#

Evaluated but not interpolated
@raven ridge yes, so in a case where it is the evaluation that is expensive, what logging does wouldn't help

raven ridge Sep 23, 2020, 12:18 AM

#

So today we save some of the cost, this would save more of the cost.

teal yacht Sep 23, 2020, 12:18 AM

#

it wouldn't really bring many benefits i'm sure, an ast node is surely more expensive than most cases where we get just another reference

grave jolt Sep 23, 2020, 12:18 AM

#

What about log(lambda: "%s" % obj)? 🙂

gleaming rover Sep 23, 2020, 12:18 AM

#

precisely

#

@grave jolt have you not ruined enough lives with chr(10).join

#

(honestly I thought of that too but you were quicker)

spark magnet Sep 23, 2020, 12:18 AM

#

@teal yacht i see, you meant we save the cost of the formatting

#

@teal yacht ast nodes are not relevant here, the compilation only happens once

gleaming rover Sep 23, 2020, 12:19 AM

#

it wouldn't really bring many benefits i'm sure, an ast node is surely more expensive than most cases where we get just another reference
@teal yacht I don't think there are many cases where it would be better, which is why I said it was a joke

teal yacht Sep 23, 2020, 12:19 AM

#

yeah, we don't want to store large strings for long durations, as we can't predict when the logger will actually flush

gleaming rover Sep 23, 2020, 12:19 AM

#

adding a whole new evaluation strategy just for this...

#

🤔

spark magnet Sep 23, 2020, 12:20 AM

#

@gleaming rover the problem isn't adding another strategy, it's adding a second strategy, with no way to know when to use which one.

teal yacht Sep 23, 2020, 12:20 AM

#

~~they literally added an operator for matching regexes in while loops so~~

gleaming rover Sep 23, 2020, 12:20 AM

#

with no way to know when to use which one.
@spark magnet what do you mean by this part?

spark magnet Sep 23, 2020, 12:20 AM

#

that was for readability.

#

@gleaming rover how would Python know whether an argument was by-name or not?

gleaming rover Sep 23, 2020, 12:21 AM

#

function definition

spark magnet Sep 23, 2020, 12:21 AM

#

this is what i mean: when the call is compiled, the func def isn't around.

teal yacht Sep 23, 2020, 12:21 AM

#

lisp does it just fine tbh

#

obviously you'd need a way to represent the ast in a sane way, and an alternative way to define macros/call-by-name functions

raven ridge Sep 23, 2020, 12:22 AM

#

that was for readability.
@spark magnet for deduplication, really. The loop-and-a-half examples were really the convincing ones, I think

spark magnet Sep 23, 2020, 12:23 AM

#

i think you quoted the wrong line, but yes.

raven ridge Sep 23, 2020, 12:23 AM

#

yes, I did. stupid mobile 🙂

gleaming rover Sep 23, 2020, 12:24 AM

#

hm I don't have enough knowledge to put what I'm thinking in the right terms

spark magnet Sep 23, 2020, 12:24 AM

#

@teal yacht my point is that the compiler would need to know for each function call what the function definition said to do with the args.

teal yacht Sep 23, 2020, 12:24 AM

#

yes, what would be the issue with that ?

spark magnet Sep 23, 2020, 12:25 AM

#

the compiler doesn't examine the imported files

teal yacht Sep 23, 2020, 12:25 AM

#

lisps do it just fine, and there exists many many lisp dialects, running on different internal mechanism

#

you can add it to the function object

gleaming rover Sep 23, 2020, 12:25 AM

#

but I'm p sure it would work?

#

yes, that

spark magnet Sep 23, 2020, 12:25 AM

#

the compiler doesn't have the function object

gleaming rover Sep 23, 2020, 12:25 AM

#

at the call site the caller knows what it's calling

teal yacht Sep 23, 2020, 12:26 AM

#

really instead of having <function object at xxx> you'd have <macro object at xxx>

spark magnet Sep 23, 2020, 12:26 AM

#

but the compiler has already reduced the expressions to bytecode

#

so now everything is a macro?

raven ridge Sep 23, 2020, 12:27 AM

#

cmd = input("Enter a command or 'quit':")
while cmd != 'quit':
    process(cmd)
    cmd = input("Enter a command or 'quit':")

^ Code like this is the compelling argument in favor of the walrus. Because the alternative without the walrus is usually something much uglier, like:

cmd = None
while cmd is not None and cmd != 'quit':
    if cmd is not None:
        process(cmd)
    cmd = input("Enter a command or 'quit':")

And with the walrus it's the much nicer:

while (cmd := input("Enter a command or 'quit':")) != "quit":
    process(cmd)

teal yacht Sep 23, 2020, 12:27 AM

#

macros inherently require a preprocessing step, so yeah

spark magnet Sep 23, 2020, 12:28 AM

#

and they require having the source of the imported modules?

teal yacht Sep 23, 2020, 12:29 AM

#

why the source ?

raven ridge Sep 23, 2020, 12:29 AM

#

import foo
foo.bar(x)

spark magnet Sep 23, 2020, 12:29 AM

#

ok, not the source, but you have to have the code in some form.

raven ridge Sep 23, 2020, 12:29 AM

#

in order to do what you're proposing, Python would need to generate different bytecode depending on whether foo.bar() is pass-by-name or pass-by-object-reference

#

and at the point where it generates the bytecode, it has no idea what bar is, or whether bar exists - it's even entirely possible that bar doesn't exist yet but will later.

spark magnet Sep 23, 2020, 12:30 AM

#

exactly

teal yacht Sep 23, 2020, 12:30 AM

#

i mean, yeah, that's because python is dumb

#

we could rework things out

#

in this very hypothetical unreasonable unwanted scenario where macros get added to python

raven ridge Sep 23, 2020, 12:30 AM

#

if you change something as fundamental as that, you get a completely different language.

teal yacht Sep 23, 2020, 12:31 AM

#

i'd say it wouldn't be a bad thing 🙃

raven ridge Sep 23, 2020, 12:31 AM

#

get a discord for your new language. 😛

#

seriously, whether such a hypothetical language would be better or not aside, there's no way to evolve Python from where it is today to that, considering backwards compatibility.

feral cedar Sep 23, 2020, 12:32 AM

#

python 5

teal yacht Sep 23, 2020, 12:32 AM

#

mandatory https://www.youtube.com/watch?v=BvECNQRrjCY

YouTube

Tsoding

Bubble Sort in Python 5

Please support I need tea https://www.patreon.com/tsoding

▶ Play video

raven ridge Sep 23, 2020, 12:37 AM

#

that said, it would be possible to make a lazy-evaluation operator, I think. So that you can do py logger.info("%s", lazy fib(100)) and it will pass a new LazyExpression object with a .get_value() method, and where the expression isn't evaluated until get_value() is called on it.

teal yacht Sep 23, 2020, 12:38 AM

#

that'd be strictly equal to wrapping into a lambda tho, except it's slightly more obvious

raven ridge Sep 23, 2020, 12:38 AM

#

right - syntactic sugar over that.

safe hedge Sep 23, 2020, 12:38 AM

#

Python as a language is syntactic sugar really though right

spark magnet Sep 23, 2020, 12:38 AM

#

what does that mean?

safe hedge Sep 23, 2020, 12:39 AM

#

The whole point is that stuff is concise and simple to do with fairly obvious names

spark magnet Sep 23, 2020, 12:39 AM

#

isn't every language in that sense sugar?

safe hedge Sep 23, 2020, 12:39 AM

#

So syntactic sugar in python is kind of the point

spark magnet Sep 23, 2020, 12:40 AM

#

what's the difference between "syntactic sugar" and "syntax"?

raven ridge Sep 23, 2020, 12:40 AM

#

foo.x is syntactic sugar for getattr(foo, 'x'), which is syntactic sugar for foo.__getattr__(foo, 'x') when foo is an object. Syntactic sugar is very powerful for guiding people towards usage that you want to encourage and away from usage that you want to discourage.

safe hedge Sep 23, 2020, 12:40 AM

#

I was merely trying to express that introducing something that was syntactic sugar around lambda like lazy would be pretty on-brand for python

spark magnet Sep 23, 2020, 12:41 AM

#

actually i think a new way to write a lambda would never be accepted, because it dosn't add anything new

raven ridge Sep 23, 2020, 12:41 AM

#

what's the difference between "syntactic sugar" and "syntax"?
@spark magnet I'll take a stab at this: "syntactic sugar" is introducing new syntax to make an operation that was already possible using existing primitives easier.

#

not all new syntax is sugar, but all sugar is new syntax.

spark magnet Sep 23, 2020, 12:42 AM

#

is multiplication syntactic sugar for repeated addition?

safe hedge Sep 23, 2020, 12:42 AM

#

Yes

raven ridge Sep 23, 2020, 12:42 AM

#

no, it isn't.

gleaming rover Sep 23, 2020, 12:42 AM

#

I don't think it is

raven ridge Sep 23, 2020, 12:42 AM

#

it has different semantics.

gleaming rover Sep 23, 2020, 12:43 AM

#

they're functionally different

spark magnet Sep 23, 2020, 12:43 AM

#

is the walrus operator syntactic sugar?

grave jolt Sep 23, 2020, 12:43 AM

#

foo.x is syntactic sugar for getattr(foo, 'x'), which is syntactic sugar for foo.__getattr__(foo, 'x')
and foo.__getattr__(foo, 'x') is syntactic sugar for getattr(foo, "__getattr__")(foo, 'x')... you get the idea 🙂

raven ridge Sep 23, 2020, 12:44 AM

#

but a*b is syntactic sugar. You could do what it does without it:

res = type(a).__mul__(a, b)
if res is NotImplemented:
    res = type(b).__mul__(b, a)
    if res is NotImplemented:
        raise ...

safe hedge Sep 23, 2020, 12:44 AM

#

2 * 4 = 2 + 2 + 2 + 2

spark magnet Sep 23, 2020, 12:44 AM

#

i think lot a lot of software terms, it's used like it's black/white, but it's really fuzzy

gleaming rover Sep 23, 2020, 12:44 AM

#

but a*b is syntactic sugar. You could do what it does without it:
res = type(a).__mul__(a, b)
if res is NotImplemented:
    res = type(b).__mul__(b, a)
    if res is NotImplemented:
        raise ...

@raven ridge isn't there a subclass check

grave jolt Sep 23, 2020, 12:44 AM

#

like compiled vs interpreted? @spark magnet

spark magnet Sep 23, 2020, 12:44 AM

#

yes 🙂

safe hedge Sep 23, 2020, 12:44 AM

#

Simple multiplication is definitely just multiple additions in disguise

gleaming rover Sep 23, 2020, 12:44 AM

#

Simple multiplication is definitely just multiple additions in disguise
@safe hedge mathematically, yes

spark magnet Sep 23, 2020, 12:44 AM

#

@safe hedge how can it be syntactic sugar if it results in different i386 instructions?

teal yacht Sep 23, 2020, 12:45 AM

#

and that's only for Nat * Nat

#

Uh, Z * Z

#

It's 3 am

safe hedge Sep 23, 2020, 12:45 AM

#

Oh I thought you meant mathematically

teal yacht Sep 23, 2020, 12:45 AM

#

even mathematically

raven ridge Sep 23, 2020, 12:45 AM

#

2 * 4 = 2 + 2 + 2 + 2
@safe hedge

In [41]: .1 + .1 + .1 + .1 + .1 + .1 + .1 + .1 + .1 + .1
Out[41]: 0.9999999999999999

In [42]: .1 * 10
Out[42]: 1.0

gleaming rover Sep 23, 2020, 12:46 AM

#

and that's only for Nat * Nat
@teal yacht okay, fine

raven ridge Sep 23, 2020, 12:46 AM

#

@raven ridge isn't there a subclass check
@gleaming rover Don't think so - what would the check do?

safe hedge Sep 23, 2020, 12:46 AM

#

I did qualify it to "simple" multiplication

raven ridge Sep 23, 2020, 12:48 AM

#

I did have a mistake there, though - type(b).__mul__ should be type(b).__rmul__

gleaming rover Sep 23, 2020, 12:48 AM

#

@gleaming rover Don't think so - what would the check do?
@raven ridge ah, never mind, it doesn't apply to __mul__

#

my mistake

raven ridge Sep 23, 2020, 12:49 AM

#

a language's data model and execution model determines what operations it allows. A language's choice of syntactic sugar determines what operations it encourages.

safe hedge Sep 23, 2020, 12:49 AM

#

When I protested that f-strings != .format I did not expect to end up here 😆

gleaming rover Sep 23, 2020, 12:50 AM

#

#internals-and-peps is just #esoteric-python without code

raven ridge Sep 23, 2020, 12:50 AM

#

if Python didn't allow attribute lookup with a.x, the fact that attributes can be stored on any object would more likely be trivia than a fundamental feature relied upon constantly by everyone.

grave jolt Sep 23, 2020, 12:52 AM

#

I would say that "syntax sugar" is something that can be transformed into "plain syntax" with a sort of preprocessing step, i.e. it's something that is exactly the same in all aspects. Like decorators:

@f
def g():
    ...
# exactly the same as:
def g():
    ...
g = f(g)
# except when you inspect the stack frame and stuff, but that doesn't count, really!

Or like do-notation in Haskell:

f = do
    x <- hello "world"
    y <- abc 'd'
    z <- f x y
    return (x, y, z)
-- exactly the same as:
f = hello "world" >>= (\x -> abc 'd' >>= (\y -> f x y >>= (\z -> (x, y, z))))

In Haskell there are many compilation steps, and on some point some syntax sugar is actually turned into simpler syntax along with other modifications.

#

In Python there isn't really such step, but I guess there could be

#

so in Python it's kind of vague

raven ridge Sep 23, 2020, 12:53 AM

#

I agree, I'm just imagining that the "preprocessing step" is something that a human could do.

#

if you can get from (new syntax) to (existing syntax) via some mechanical transformation, and the (new syntax) is designed to be nicer to use than (existing syntax), then (new syntax) is syntactic sugar for (existing syntax)

#

regardless of whether the language itself has any sort of mechanism to enable that preprocessing step.

spark magnet Sep 23, 2020, 12:55 AM

#

i guess the problem is "the step" has gradations of difficulty, and the before and after have gradations o fidelity.

grave jolt Sep 23, 2020, 12:56 AM

#

Is yield from syntactic sugar for

_i = iter(EXPR)
try:
    _y = next(_i)
except StopIteration as _e:
    _r = _e.value
else:
    while 1:
        try:
            _s = yield _y
        except GeneratorExit as _e:
            try:
                _m = _i.close
            except AttributeError:
                pass
            else:
                _m()
            raise _e
        except BaseException as _e:
            _x = sys.exc_info()
            try:
                _m = _i.throw
            except AttributeError:
                raise _e
            else:
                try:
                    _y = _m(*_x)
                except StopIteration as _e:
                    _r = _e.value
                    break
        else:
            try:
                if _s is None:
                    _y = next(_i)
                else:
                    _y = _i.send(_s)
            except StopIteration as _e:
                _r = _e.value
                break
RESULT = _r
```?

raven ridge Sep 23, 2020, 12:56 AM

#

I'd argue that if it loses fidelity it's not sugar - it's only sugar if there's already a way to do exactly the same thing.

#

but I don't draw any such distinction for difficulty - whether it's a huge, involved, multi-step transformation to get from new to old, or it's a simple substitution, I'd call both of those sugar for the old syntax.

#

Is yield from syntactic sugar for
IMHO, yes. It's not something new in the data model, it's something that was always possible but that now has blessed and accessible syntax, encouraging its use.

#

(assuming you copied that from the yield from PEP, at least - I didn't actually read it, I just think that yield from is syntactic sugar for something, and that looks like it might be it 😆 )

grave jolt Sep 23, 2020, 1:02 AM

#

Yeah, I copied it

#

there's no way I'm writing this by hand 🙂

covert saddle Sep 23, 2020, 1:39 AM

#

class MyClass:
    my_attr: str
   
    def my_method(self):
        pass

Is there any way that I can access the type-hint of my_attrwithin my_method?

teal yacht Sep 23, 2020, 1:43 AM

#

self.__annotations__

safe hedge Sep 23, 2020, 1:48 AM

#

If I have a module/pkg and I want to use it's string name (i.e. 'mypkg.module') and then also import it, is it a bad or OK idea to do something like the following:

from importlib import import_module

MY_MODULE = 'mypkg.mymodule'

def somefunc():
   mods = [name for _, name, is_pkg in pkgutil.iter_modules([import_module(MY_MODULE).__path__[0]])]

#

The alternative would be to wither hardcode the name as a string where I use it or to use:

import mypkg.mymodule

def somefunc2():
    do_stuff(mypkg.mymodule.__name__)

raven ridge Sep 23, 2020, 1:54 AM

#

I like the last option best. It's succinct, and immediately obvious to the reader.

#

Or just use "mypkg.mymodule" instead of mypkg.mymodule.__name__

#

but if you really don't want it as a string literal, your last option seems nice to me.

safe hedge Sep 23, 2020, 1:56 AM

#

It's more that if I were to change the path to the module then I could see having string representations hardcoded being a likely source of bugs

#

At least if I do mypkg.mymodule.__name__ and then changed the import to mypkg.mymodule.submodule or whatever there's going to be a very clear errort

covert saddle Sep 23, 2020, 1:57 AM

#

self.__annotations__
@teal yacht ok thanks

raven ridge Sep 23, 2020, 1:57 AM

#

You'd wind up find/replacing the module name, and it wouldn't be too tough to match the string at the same time.

#

but yeah, the __name__ approach is fine by me.

safe hedge Sep 23, 2020, 1:58 AM

#

True I guess, thanks. I always feel uneasy using double underscore variables because I worry I've misunderstood what they'll be

#

Like I wasn't sure if __name__ would always return what I expected

raven ridge Sep 23, 2020, 1:59 AM

#

As far as I know, it will. __name__ is always the name by which a module was imported, except for the initial module which is always named "__main__"

safe hedge Sep 23, 2020, 2:05 AM

#

It depends what you mean by "the name by which it was imported" I guess. Because os.path returns posixpath which I guess could throw some people

raven ridge Sep 23, 2020, 2:06 AM

#

true enough.

#

and it is possible to assign to __name__ too.

safe hedge Sep 23, 2020, 2:12 AM

#

I'm just going to go ahead and assume that would be a very bad idea

raven ridge Sep 23, 2020, 2:22 AM

#

I've done it for things defined in private submodules and re-exported by their parent package... But mostly to work around the fact that Sphinx handles private submodules poorly.

red solar Sep 23, 2020, 3:34 AM

#

lol

wide shuttle Sep 23, 2020, 5:41 AM

#

@left bronze I'm not quite sure what the relevance of that was for the channel. It looked more like advertisement than something that directly discussed the Python programming language itself.

left bronze Sep 23, 2020, 5:42 AM

#

Found the info, downloaded the image and shared over here

#

Not aware that sharing pic like the one is not allowed

#

@wide shuttle

#

I'll type it out next time onwards

wide shuttle Sep 23, 2020, 5:50 AM

#

It's more that this channel is for discussing the Python programming language itself, from a higher-level and more abstract perspective. The image that you shared seemed a bit random and did not relate to a discussion of Python itself as far as I can tell, @left bronze.

left bronze Sep 23, 2020, 5:51 AM

#

Oh ok
Got it
The image I shared was telling an alternative of pandas that helps in scalability
But cool, I'll keep it in mind before sharing

safe hedge Sep 23, 2020, 5:52 AM

#

Probably would be something better suited to #data-science-and-ml

#

Does anyone have good explanation of how the ispkg value in ModuleInfo returned by pkgutil.iter_modules is calculated? I tried following the source code but I got a little lost at sys.path_importer_cache which is written in c

craggy sinew Sep 23, 2020, 5:56 AM

#

Yo would you guys know a way to reset a function

safe hedge Sep 23, 2020, 5:58 AM

#

Need more details than that, maybe open a help channel #❓｜how-to-get-help

willow obsidian Sep 23, 2020, 6:25 AM

#

who all are game developers (indie)

#

pls reply

covert saddle Sep 23, 2020, 6:30 AM

#

class MyClass:
   def my_method(arg1: str, arg2: int):
      pass

Is this any way I can access the type hint of args1 and arg2 inside my_method?

#

I tried arg1.__annotations__ but it doesn't work

grave jolt Sep 23, 2020, 6:30 AM

#

Well, why would the variable arg1__annotations__ be defined?

#

You can access my_method.__annotations__

#

arg1.__annotations__ can't possibly work: arg1 is a string and therefore is immutable

covert saddle Sep 23, 2020, 6:31 AM

#

ayye thanks it works

#

Yeah I get it

grave jolt Sep 23, 2020, 6:31 AM

#

annotations belong to a function, not the value

#

👍

covert saddle Sep 23, 2020, 6:31 AM

#

Does annotations mean anything?

#

actually leave it, ill research it

grave jolt Sep 23, 2020, 6:32 AM

#

Annotations don't change anything at runtime, except appearing in __annotations__

covert saddle Sep 23, 2020, 6:33 AM

#

ok thanks again

unkempt rock Sep 23, 2020, 7:08 AM

#

Is it possible to create my own cryptocurrency using python? lmao

dry tartan Sep 23, 2020, 7:08 AM

#

@unkempt rock https://www.freecodecamp.org/news/create-cryptocurrency-using-python/

freeCodeCamp.org

How to Create Your Own Cryptocurrency Using Python

With the current rise of cryptocurrencies, blockchain is creating a buzz in the technology world. What if the freeCodeCamp could also enter this craze and create its own digital currency?

covert saddle Sep 23, 2020, 7:25 AM

#

Is there any way I can pass all the arguments to a new function without having to specify them explicitly?
For example:

def my_func(arg1, arg2, ...):
  print("something here which will print the value of arg1, arg2 and the others without me having to specify them explicitly?")

gleaming rover Sep 23, 2020, 7:27 AM

#

Is there any way I can pass all the arguments to a new function without having to specify them explicitly?
For example:
def my_func(arg1, arg2, ...):
  print("something here which will print the value of arg1, arg2 and the others without me having to specify them explicitly?")

@covert saddle if you have explicit parameter names, I don't think so

#

at least, not without some magic

covert saddle Sep 23, 2020, 7:27 AM

#

yeah *args, **kwargs would have been easy but it's not the case here

gleaming rover Sep 23, 2020, 7:27 AM

#

if you're not using type hints, though, how about copy-paste? 🙂

covert saddle Sep 23, 2020, 7:28 AM

#

wdym?

gleaming rover Sep 23, 2020, 7:28 AM

#

copy-paste the parameter names from the function signature 🤔

covert saddle Sep 23, 2020, 7:29 AM

#

oh but I have to call this for a lot of fucntions. I was searching for something programmatic.

#

if you're not using type hints, though, how about copy-paste? 🙂
@gleaming rover What if I apply type hints to them? def my_func(arg1: str, arg2: int): pass. Would it be possible then?

gleaming rover Sep 23, 2020, 7:31 AM

#

no, if you're using type hints then you can't even copy-paste

#

anyway, I wouldn't recommend this, but...

#

>>> import inspect
>>> def f(a, b, c, *args, **kwargs):
...     args = inspect.currentframe().f_locals
...     print(args)
... 
>>> f(1, 2, 3, 4, 5, f=6, g=7)
{'a': 1, 'b': 2, 'c': 3, 'args': (4, 5), 'kwargs': {'f': 6, 'g': 7}}

covert saddle Sep 23, 2020, 7:33 AM

#

Well basically I am checking if all the values specified to a function are among a list of permitted values. If they are not, I am raising an error so that they don't misuse the function and fuck up their system. is there a better way of doing this than what I am trying to do?

gleaming rover Sep 23, 2020, 7:33 AM

#

are there patterns in the ranges of accepted values

#

if there are you could use a deco

covert saddle Sep 23, 2020, 7:34 AM

#

>>> import inspect
>>> def f(a, b, c, *args, **kwargs):
...     args = inspect.currentframe().f_locals
...     print(args)
... 
>>> f(1, 2, 3, 4, 5, f=6, g=7)
{'a': 1, 'b': 2, 'c': 3, 'args': (4, 5), 'kwargs': {'f': 6, 'g': 7}}

@gleaming rover Yeah I saw this on SO, but it said not recommended too

gleaming rover Sep 23, 2020, 7:34 AM

#

hard to say without knowing the details

#

but I think what you have is more an architecture problem than an implementation problem

covert saddle Sep 23, 2020, 7:34 AM

#

I am not sure what that means

safe hedge Sep 23, 2020, 7:34 AM

#

Can you just give the example code you have and what you are trying to do? Might be best in a help thread though

gleaming rover Sep 23, 2020, 7:35 AM

#

in other words, you're asking "how do I do this?", when you might need to ask "what should I change so I won't have to do this?"

covert saddle Sep 23, 2020, 7:35 AM

#

the general thread or help-* one?

Can you just give the example code you have and what you are trying to do? Might be best in a help thread though
@safe hedge

merry pollen Sep 23, 2020, 7:35 AM

#

is there a particular reason to do f_locals instead of just locals()

gleaming rover Sep 23, 2020, 7:35 AM

#

wups

#

forgot all about locals()

merry pollen Sep 23, 2020, 7:35 AM

#

it's the same thing on the first line right

gleaming rover Sep 23, 2020, 7:35 AM

#

yeah, you're right

covert saddle Sep 23, 2020, 7:35 AM

#

is there a particular reason to do f_locals instead of just locals()
@merry pollen That also gives the values of variables inside the fuction, not just the arguments. I don't really require the non arguments variables.

merry pollen Sep 23, 2020, 7:35 AM

#

locals().keys() ?

gleaming rover Sep 23, 2020, 7:36 AM

#

@covert saddle my thought process was that you would call it first

merry pollen Sep 23, 2020, 7:36 AM

#

ohhh wait

#

you want to check before the function is run? basically?

#

boy do i have the code for you

covert saddle Sep 23, 2020, 7:36 AM

#

yeah basically

gleaming rover Sep 23, 2020, 7:36 AM

#

@merry pollen That also gives the values of variables inside the fuction, not just the arguments. I don't really require the non arguments variables.
@covert saddle no, that's not the distinction

merry pollen Sep 23, 2020, 7:36 AM

#

i have a decorator for just this,.... somewhere

#

lemme find it

gleaming rover Sep 23, 2020, 7:36 AM

#

>>> def f(a, b, c, *args, **kwargs):
...     h = 8
...     print(locals())
...     print(inspect.currentframe().f_locals)
... 
>>> f(1, 2, 3, 4, 5, f=6, g=7)
{'a': 1, 'b': 2, 'c': 3, 'args': (4, 5), 'kwargs': {'f': 6, 'g': 7}, 'h': 8}
{'a': 1, 'b': 2, 'c': 3, 'args': (4, 5), 'kwargs': {'f': 6, 'g': 7}, 'h': 8}

covert saddle Sep 23, 2020, 7:37 AM

#

Hmm they both do the same thing then. This is not what I want.

gleaming rover Sep 23, 2020, 7:37 AM

#

the idea is

#

you would run it as the first line

#

in the function

merry pollen Sep 23, 2020, 7:38 AM

#

you need a decorator that will intercept arguments and check them with the decorated function's signature

gleaming rover Sep 23, 2020, 7:38 AM

#

you could also use a decorator to capture all arguments

merry pollen Sep 23, 2020, 7:38 AM

#

if you want checking-before-function-evaluation

covert saddle Sep 23, 2020, 7:38 AM

#

you need a decorator that will intercept arguments and check them with the decorated function's signature
@merry pollen yeah that'd be great

safe hedge Sep 23, 2020, 7:38 AM

#

I mean a help- thread

#

I still think it's hard to know what is best for you without any code

merry pollen Sep 23, 2020, 7:39 AM

#

this is like, perhaps an example of what you want

#

where it basically does a check that a certain set of the function's arguments are either all provided or none of them are provided

covert saddle Sep 23, 2020, 7:39 AM

#

@safe hedge moved to #help-peanut

covert saddle Sep 23, 2020, 9:41 AM

#

@opaque etherRead #❓｜how-to-get-help . Then tag me when you claim a channel.

void ocean Sep 23, 2020, 9:56 AM

#

Hi

loud summit Sep 23, 2020, 10:01 AM

#

This is not a help channel, if you need help use #❓｜how-to-get-help @void ocean

unkempt rock Sep 23, 2020, 11:57 AM

#

Is it possible to activate a mining program that mines bitcoin using python

#

No downloads needed, just code

#

and uses the cpu or gpu

radiant fulcrum Sep 23, 2020, 12:40 PM

#

I mean you could do anything just whether or not it's a good idea is another thing

#

Also not the channel for this

heady mauve Sep 23, 2020, 1:59 PM

#

what is the hex representation of the LOAD_CONST bytecode as stored in a pyc file? Python 3.8.2

pallid meteor Sep 23, 2020, 2:01 PM

#

This isnt a help channel guys, its a discussion channel about the python language and its features

#

If youre trying to get help, #❓｜how-to-get-help

full jay Sep 23, 2020, 2:04 PM

#

I asked them to come to here since it was something specific about implementation

#

Apologies

last pollen Sep 23, 2020, 2:16 PM

#

what is the hex representation of the LOAD_CONST bytecode as stored in a pyc file? Python 3.8.2
that's a valid question for this channel.

#

@heady mauve It should be 64 hex, 100 dec, d ASCII

#

!e

import dis
print(dis.opname.index("LOAD_CONST"))

fallen slateBOT Sep 23, 2020, 2:21 PM

#

@last pollen :white_check_mark: Your eval job has completed with return code 0.

heady mauve Sep 23, 2020, 2:32 PM

#

@last pollen got it, thanks!

#

when pyc files are compiled, do they contain a timestamp? I compiled the same source twice with pycompile and I found that bytes 08 and 09 were different. They were followed by three bytes which stayed constant. CPython 3.8.2 on Linux.

full jay Sep 23, 2020, 2:40 PM

#

It does yeah

#

One of the things it checks is whether or not the file you're wanting to compile is newer than or was recently edited compared to your current .pyc file or if the file has changed

#

It's typically faster for it to just see "oh the source file had an edit at 5:30 and the compiled file was made at 5:15." I think that's how it works anyway

#

Now I'm second guessing myself

heady mauve Sep 23, 2020, 2:44 PM

#

thanks!

grave jolt Sep 23, 2020, 3:12 PM

#

well, I believe that's how makefile works as well, kind of

#

so it could work

full jay Sep 23, 2020, 3:38 PM

#

Yeah I just can't remember if it's purely based on time or if it hashes the file as well

#

I would think hashing it and comparing would be costly

mint forge Sep 23, 2020, 3:45 PM

#

i got some quesions about tests is it suited for this channel

full jay Sep 23, 2020, 3:48 PM

#

I think that'd be better for #unit-testing

hollow tangle Sep 23, 2020, 3:54 PM

#

what is the best practice for having environment variables in an application.

#

I have windows and i usually use the setx command

tender ingot Sep 23, 2020, 4:18 PM

#

@worldly venture Can you unban The Python Teacher