internals-and-peps

[unkempt rock]

how do i make keywords for python?

[amber nexus]

You don't

[unkempt rock]

how abt

async

and await

[amber nexus]

Those are built into the language

You can't create custom keywords

[unkempt rock]

oh

sad

how does python create a variable?

xD

[raven surge]

In [13]: from typing import List

In [14]: def create_graph_0(nums) -> dict:
    ...:         g = {}
    ...:
    ...:         # this wrapper func solves the issue with create_graph_1
    ...:         def gen_children(node, end):
    ...:             return (node + j for j in range(1, end + 1))
    ...:
    ...:         for node in range(0, len(nums) - 1):
    ...:             g[node] = gen_children(node, nums[node])
    ...:
    ...:         g[len(nums) - 1] = (None for i in range(0, 1))
    ...:
    ...:         return g
    ...:

In [15]: def create_graph_1(nums) -> dict: # uses gen expr within for loop (bad idea 💩)
    ...:         g = {}
    ...:
    ...:         for node in range(0, len(nums) - 1):
    ...:             g[node] = (node + j for j in range(1, nums[node] + 1)) #what is the value of node for each gen expr?
    ...:
    ...:         g[len(nums) - 1] = (None for i in range(0, 1))
    ...:
    ...:         return g
    ...:

In [16]: def can_jump_gh(nums: List[int], generate_graph_func) -> bool:
    ...:         """Uses BFS to check if we can reach end node -> nums[-1]"""
    ...:         g = generate_graph_func(nums)
    ...:         q = [(0 for i in range(0, 2))]
    ...:         seen = set()
    ...:         
    ...:         while q:
    ...:             nodes = q.pop()
    ...:             for node in nodes:
    ...:                 if node == None:
    ...:                     return True
    ...:
    ...:                 if node not in seen and node in g:
    ...:                     seen.add(node)
    ...:                     gen = g[node]
    ...:                     q.append(gen)  # visit children later
    ...:
    ...:         return False
    ...:

In [17]: can_jump_gh([2,3,1,1,4], generate_graph_func=create_graph_0)
Out[17]: True

In [18]: can_jump_gh([2,3,1,1,4], generate_graph_func=create_graph_1) # some times it seems to work 
Out[18]: True

In [30]: can_jump_gh([1,0,1,0], generate_graph_func=create_graph_0)
Out[30]: False

In [31]: can_jump_gh([1,0,1,0], generate_graph_func=create_graph_1) # wrong
Out[31]: True

Mutation bugs are the worst, and it's so easy to accidentally have them in your code. For example create_graph_0 and create_graph_1, create_graph_1 has a bug.

[brave badger]

!e
Names in Python are late-binding, and specifically in that example since generator expressions are lazily evaluated, you get behavior such as:

xxs = []

for start in range(3):
    xxs.append((i for i in range(start, 5)))

for xs in xxs:
    print(*xs)

[fallen slate]

@brave badger :white_check_mark: Your eval job has completed with return code 0.

001 | 0 1 2 3 4
002 | 1 2 3 4
003 | 2 3 4

[brave badger]

@raven surge In fact, since for loops leave the last start still assigned, I think you can del that and break the next loop. Checked, doesn't seem to be the case, but still is interesting behavior

[raven surge]

@brave badger

In [54]: nums = [1,0,1,0]

In [55]: g2 = {}

In [56]: for node in range(0, len(nums) - 1):
    ...:      g2[node] = (node + j for j in range(1, nums[node] + 1)) # issue
 
In [57]: def gen_children(node, end):return (node + j for j in range(1, end + 1))

In [58]: g1 = {}

In [59]: for node in range(0, len(nums) - 1):
    ...:      g1[node] = gen_children(node, nums[node] + 1)

In [60]: list(g2[0])
Out[60]: [3]

In [61]: list(g1[0])
Out[61]: [1, 2]

In [62]:

Simpler way to reproduce the issue. You are correct lazy evaluation bounds values passed to it at execution time.

[peak spoke]

The iterable expression in the genexp has to be evaluated to get the iterator so the names don't matter there

But after a couple of problems with the late binding closures you'll be able to spot whhat the problem is fairly easily

[spice pecan]

the node + j part is probably what fails

since node is getting reassigned each iteration

[peak spoke]

The closure is linked to the name instead of the object, so it looks up the node name when evaluated which only has the one value

[fallen slate]

Hey @unkempt rock!

It looks like you tried to attach file type(s) that we do not allow (.exe). We currently allow the following file types: .3gp, .3g2, .avi, .bmp, .gif, .h264, .jpg, .jpeg, .mkv, .mov, .mp4, .mpeg, .mpg, .png, .tiff, .wmv, .svg, .psd, .ai, .aep, .xcf, .mp3, .wav, .ogg, .webm, .webp, .flac, .afdesign, .m4a, .csv.

Feel free to ask in #community-meta if you think this is a mistake.

[narrow kettle]

rip

[grave jolt]

I'm trying to understand what a free monad is, and so far I failed miserably at that.

Are Python generators a kind of a free monad?

[raven ridge]

As far as I can see, no... A free monad is a type of monad, and a monads

[provide] their own data type (a particular type for each type of monad), which represents a specific form of computation, along with one procedure to wrap values of any basic type within the monad (yielding a monadic value) and another to compose functions that output monadic values (called monadic functions).

Generators do not, in general, fit that description, do they?

[brave badger]

Coroutines are monadic, although I'm not too sure about generators

[brave badger]

@unkempt rock @sly narwhal Please refrain from posting off-topic clutter in this channel. The channel topic states that it's for "Discussion on the use cases, implementation and future of the Python programming language including PEPs, advanced language concepts, new releases, the standard library, and the overall design of the language.".

[bitter abyss]

hey i have one optional argument

i am using argparse

like -o

now i want want user to enter only hi or bye

is there a way ?

[undone hare]

Hey @bitter abyss, see #❓｜how-to-get-help

[bitter abyss]

@undone hare ppl takes a lot time there to reply bro

[lofty rover]

My bro, people are doing it for free.

[bitter abyss]

yeah ik

[grave jolt]

Well, they fit this description if the monad "provides one or zero values", then this:

@some_magic_decorator
def my_gen(x):
    y = yield foo(x)
    z = yield bar(x, y)
    return Pure( (x, y, z) )

will do the same as this:

def my_gen(x):
    return (
        foo(x)
        .then(lambda y:
            bar(x, y)
            .then(lambda z:
                Pure( (x, y, z) )
        )
    )

so it's kind of a monad

[swift imp]

You can use yield like that?

[flat gazelle]

Yield is an expression, yes

[swift imp]

If you iterate my_gen does it eventually use the return?

[flat gazelle]

@proper remnant this is not a help channel, please read #❓｜how-to-get-help

[swift imp]

Learning new things every day

[boreal umbra]

class ThingMaker:
    def thingy() -> ThingType: ...
    def mabob() -> MabobType: ...

Is there a way to specify what types ThingType and MabobType are for that particular instance, how would you do that? For example, if you had tm: ThingMaker[int, str] = ..., how would you make that information usable for the linter?

Or perhaps that behavior isn't supported. I'm not aware of any "compound types" for non-containers.

[peak spoke]

I'm not sure what exactly you need but that sound like a Generic would fit

[grave jolt]

Can you elaborate? not sure what you want.

class Bifactory(Generic[A, B]):
    @classmethod
    def thingy(cls) -> A: ...

    @classmethod
    def shmingy(cls) -> B: ...

You mean this?

[boreal umbra]

For tm: ThingMaker[int, str] = ... to have the behavior that a type checker will now expect ThingType to be int and MabobType to be str for that instance.

[grave jolt]

then you want generics^, yes

[boreal umbra]

So having Generic[A, B] in the... idk what you call that part of the class declaration. It gives you that behavior? Cool 👍🏻

[grave jolt]

Yes, that's how all generic types are defined.

A generic type is just a type parametrized by concrete types

[boreal umbra]

I'm learning

[swift imp]

can you explain this some more? Does passing that Generic[A, B] in the class def init allow you to use the things parameterizing that Generic inside the method definitions of the class?

[grave jolt]

Well, Generic is for type checking.
It allows you to parametrize a type with other type(s)

The types are available at runtime if you created the class as Class[int, str](...), but you don't usually instantiate a class like that

[swift imp]

wait so, is this wrong?

class foo:
  def thingy(self) -> int: ...

Like if I did that, would mppy not know what to do ? I"m confused on why you have to do that

Or maybe

class foo:
  pass

class bar:
  def thing(self) -> foo: ...

Or am I missing this big time

[raven ridge]

that's not wrong, it's just doing something entirely different. Generics are for a case like a dictionary - a dict usually has one type as its keys and a different type as its values, and some methods return keys, and some methods return values, and some methods return key/value pairs. Generic lets you explain to mypy that the type a dict method returns depends on the type of the dict itself.

for instance:

from typing import TypeVar, Generic

K = TypeVar('K')
V = TypeVar('V')

class MyDict(Generic[K, V]):
    def add_entry(self, key: K, val: V) -> None:
        ...
    def get_value(self, key: K) -> V:
        ...

[boreal umbra]

Are we getting the vibe that patma isn't going to happen? It looks like the spec is too divisive even among those who want it.

I'm looking at aeros's comment history. I guess the next steering council votes on it.

[raven ridge]

I've been getting the impression that it's likely to happen... I think some SC members started off more negative than they are now, at least

PEP 634 seems to be much better liked than PEP 622 was, at least at the start

[swift imp]

They still seem so all over the place with syntax stuff, and a ton of discussions on constants and stuff

[unkempt rock]

If I have a random file .exe/docx/txt/mp4/... can I view this file in text format?

[safe hedge]

This is not a help-channel. Please go to #❓｜how-to-get-help to find the appropriate place for your question

[peak spoke]

you could use a memoryview

[flat gazelle]

Could also make your own reference slice class, something likeRefSlice([2,5,4,4])[2:-1] (or just pass a slice object into the initializer)

[rose stream]

I have an async function that returns a class, would it be cleaner to run the function within the class init and call the class itself instead?

user = await find_or_create_by_discord_id(**extract_discord_user_data_from_context(ctx))

vs

user = User(ctx)

Or am I just wasting my time thinking about the implementation

[unkempt rock]

explain me how this word

n1=input('select number')
n2=input('select number')
starting=1
while n1<=n2:
  print(n1,'x',n2,'=',n1*n2)
  starting=starting+1

i want to multiply

pls

[peak spoke]

You pass around references to objects for everything in python

[unkempt rock]

i didn't understand

[peak spoke]

(object objects, not {} dict "objects"`)

That was as a response to @unkempt rock , you'll want to grab a help channel as described in #❓｜how-to-get-help if you need help with some code

[unkempt rock]

can you pls remake above code?

[peak spoke]

python will never copy for you, if you pass list [] somewhere, that somewhere will get an another reference to the same object

There are no primitives in python, the ints are objects that represent arbitrarily large decimals. The tuple holds references to the immutable ints inside, and when you pass the tuple it'll again only create a new reference to the already existing object. But if you do any operation that changes the tuple it'll create a new one

[spice pecan]

Yeah, every type is a reference type, assignment never copies, and only references can be passed, not names

[raven ridge]

This is all part of Python's "everything is an object" data model. There are no primitives, even int and float are objects, and they follow the same rules as passing instances of Object around in C#. Absolutely everything is passed by reference, all the time.

[raven ridge]

!e ```py
import codecs
print(codecs.decode(memoryview(b"hello world")[6:], "utf-8"))

[fallen slate]

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

world

[raven ridge]

codecs.decode() works on any bytes-like object, including a memoryview of bytes.

I don't think you're reading the code right.

What do you mean by "intermediate buffer"?

It creates a Py_buffer object, but that doesn't do any copying

A Py_buffer of a C contiguous memoryview is just a fat pointer. It's a fixed size struct that contains a pointer and a length, plus a PyObject* pointing to the object that owns the memory it's pointing into

[peak spoke]

That looks like it behaves as expected to me

[raven ridge]

What do you mean by "edge cases"?

If you mean "are there cases when a name from an outer scope can't be accessed because it is shadowed by a name from an inner scope", the answer is "yes", but given that the inner scope can choose its own names that isn't really a big deal. If you want access to the original name you can just assign it to a new name or pass it into the inner function.

[tidal marten]

Is this PEP implemented in Python 3.x?
https://www.python.org/dev/peps/pep-0636/

[grave jolt]

No. It's not even clear if it's going to happen.

[barren thunder]

Can someone help me with my code?

[mint forge]

See #❓｜how-to-get-help @barren thunder

[tidal marten]

Ohh, well... I thought it was a sure thing...

[raven ridge]

the steering council has said that they won't vote on it until after the next round of elections

Voting will end in mid December, and the next steering council will be making the call on this PEP, probably in January.

[tidal marten]

Would there be Python 4.0 in the near future, do you think?

Or is it going to be 3.10+

[raven ridge]

the next version will be 3.10

[silver edge]

you can always keep track of it at the cpython github repo.

[sacred yew]

@dusky maple #❓｜how-to-get-help

[dusky maple]

thanks

[grizzled bramble]

lol i was worried about python 4

big relief

[sacred yew]

python 4 will break some python 2/3 compatiblity checks

that are sys.version_info[0] == 3

[unkempt rock]

what was 1 to 2 like?

i wasnt around in python 1

[cloud crypt]

was pretty calm and not as breaking as python 2 to python 3

[desert peak]

also no one was using python 1 really

it was a college research project essentially

[unkempt rock]

who uses python1

1996

https://docs.python.org/release/1.5/tut/tut.html

[raven ridge]

Yes, but those checks are wrong and should be fixed. FWIW I've heard that Windows moved from Windows 8 to Windows 10, skipping 9, in part because lots of old programs had code that checked if the Windows version started with 9 to enable a legacy 95/98 compatibility setting.

[spark magnet]

that is why they skipped Windows 9

[raven ridge]

I've heard it was partly that and partly that the marketing people didn't want a lower version number than Apple had, heh (though most people I know pronounce OS X as "oh-ess-ex" instead of "oh-ess-ten")

[fallen turret]

wait, isn't 636 basically a switch-case statement? I thought that was already proposed and rejected on multiple occasions.

[raven ridge]

634, 635, and 636 are a rewritten and replacement for 622. It's not really a switch/case statement - a switch doesn't do any assigning, like pattern matching does.

[fallen turret]

oh cool

[unkempt rock]

I think the numbers are really important. People first look at the desktop view and the number. Most people choose from looking at that two factors XD

[bitter abyss]

can someone please help me in #help-chestnut

[sharp plover]

It's really more about the pattern matching than the switch statement. If you've used a language like Haskell then you'll understand that structural pattern matching is a powerful paradigm.

You'd use it to break down a larger structure into its constituent parts and assign them to variables.

[grave jolt]

In some cases, pattern matching tremendously simplifies the destructuring logic: https://github.com/decorator-factory/python-fnl/commit/8f877c94fb98408f551e6a7f6b32213e2e44b0cd?branch=8f877c94fb98408f551e6a7f6b32213e2e44b0cd&diff=split#diff-9cf42d8a3e3dc179388790be4fa8e897dbec989ac3740d7960e2c6dddad6899cL90

[unkempt rock]

as a non apple user i just now realized the x was supposed to stand for 10

[lusty schooner]

lol

[prisma kelp]

Whats actaully going on under the hood with heirarchical / nested lists

like if you want to turn a list of 2 100 item list into 100 two item lists why does that seem to generally require making a function that assigns each element to the new structure one by one

why isnt this something that can be done just by changing meta data in the list?

[peak spoke]

Python lists just store references to other objects, nested lists are the same thing as lists of integers for example, just contain the lists instead of numbers

[prisma kelp]

im aware that its all pointers under the hood, i guess im more confused about using interprert overhead to loop through everything and reassign it rather than "invert list of lists" being something implemented in C under the hood?

[peak spoke]

It's not really something worth implementing some special behaviour for as the lists are for general items. There are libs like numpy that have these kinds of optimizations

[prisma kelp]

ah ok so a conscious design decision that makes sense

[peak spoke]

numpy is fairly old and mature, so implementing it on the list type would probably be a bit redundant as it can't have the other optimizations numpy provides without being bloated

[frail hazel]

any using python with verilog ???

anyone

[spark magnet]

@prisma kelp btw: zip will do the operation you want

[pseudo cradle]

Is there anything wrong with nested dicts?

Or is there a better approach?

[grave jolt]

What's wrong with nested dicts?
If what you need is a key-value map where a value is another map, it's fine

[pseudo cradle]

I guess dict[key1][key2] could be misinterpreted as an array?

PyCharm is yelling at me for having nested dicts so I was wondering if I was being bad or if it was

[grave jolt]

How is it yelling? Can you show the code?

not the right channel, though

you should open a help channel

[pseudo cradle]

Okay

[grave jolt]

Please ask in #data-science-and-ml or in a help channel. This channel is for meta-discussions about the Python language.

[swift imp]

Here's a question, at the class def level when and how is __eq__ used? I did some metaprogramming to compare class defs for a class factory function and it seems wonky

The repl in ipython wasn't working unless I overroad __hash__ too

This was also a data class with a different metaclsss

[raven ridge]

What do you mean by "at the class def level"?

A class's __eq__ is used to compare instances of the class. A metaclass's __eq__ is used to compare the class itself.

[grave jolt]

@whole obsidian Don't advertise your help channel, especially in strictly on-topic channels like this one. If someone wants to help, they will browse the help channels.

[stuck walrus]

What does complex iterators mean vs normal iterators

[grave jolt]

It's not a separate term, I guess they just meant that you can compose and transform iterators.

[stuck walrus]

https://www.geeksforgeeks.org/python-itertools/

I was reading about infinite iterators from this link, and they sound awfully similar to generators @grave jolt

whats the difference?

[sacred yew]

generators dont have to be infinite

and infinit iterators dont have to be generators

for instance, __iter__

[grave jolt]

Generators are a subtype of iterators, but with additional functionality. Basically, a generator is a resumable function.

[unkempt rock]

You can think of an iterator as basically a kind of "abstract type"

Or rather, of abstract metaclass, you should read this, it will open your mind to a lot of things you can do in regards to defining custom data structures https://docs.python.org/3/library/collections.abc.html

[raven ridge]

A generator is a type of iterator. All generators are iterators, but not all iterators are generators. And every generator could instead be written as a class with custom __iter__ and __next__ methods that saves state in its instance variables.

[brave badger]

@unkempt rock We will not help you with graded coursework

!rule 5

[fallen slate]

Rules

5. Do not provide or request help on projects that may break laws, breach terms of services, be considered malicious or inappropriate. Do not help with ongoing exams. Do not provide or request solutions for graded assignments, although general guidance is okay.

[fossil ginkgo]

I'm fairly new to typing in python and I noticed that the python's interpreter completely ignores the types,

is there a way to enforce typing? or do I need to lint my code with mypy everytime?

(not sure if linting is the right word to describe mypy)

[unkempt rock]

no

you can use type hinting in later python versions but it only works as hints inside IDEs

if you are using dirty data then you should enforce it by yourself

[flat gazelle]

@fossil ginkgo you have to run with an external type checker like mypy or pyright to get type checking

[candid stump]

is there any way to call gRPC api's with Postman?

[fossil ginkgo]

so no way to enforce it on the interpreter's level?

[clear wadi]

Hello guys im making my own py module and I want to know, if I have a function which enables tremalloc, will that apply for the client program?

reply with @tt on

[brave badger]

Not really, you could probably enforce them through a decorator or the like though, but at that point if you have well-annotated code, you would not be at fault if the user tries to pass an invalid type and breaks everything

[fossil ginkgo]

thanks 🙂

[tidal marten]

So I was working with pyqt and notice that I cannot do abstract classes while subclassing QObject

Inheriting from QObject and an abstract class also didn't work

I guess I'm using raise NotImplementedError() instead, lol

[cloud wren]

Hey so I am on edabit just doing some coding challenges I keep seeing people do this on a for loop.
def number_length(num):
return sum(1 for i in str(num))

how do i find the name or documentation for having a "1" before the for loop? I understand the concept it is counting for every letter in num. but is it a overload of "sum" or of "for".

[unkempt rock]

@cloud wren Hey, I think #python-discussion would be fine for this question, but to answer that anyways, they’re using a generator expression, https://www.python.org/dev/peps/pep-0289/

[cloud wren]

thank you for the reply @unkempt rock , It seemed to get lost in general. I didn't know if it was a non standard thing since it is a modification of the normal way for loops are done and in that case it was more for the advanced channel.

[unkempt rock]

All good, this channel is more oriented around discussion about specific things in the language than more for help

[crude turret]

do python threads share the same callstack as the main thread, or do they have a separate one?
couldn't find anything on the documentation page for threading

they're probably referring to python thread's being subject to the language's GIL

don't quote me on that though

In computer science, a computer is CPU-bound (or compute-bound) when the time for it to complete a task is determined principally by the speed of the central processor: processor utilization is high, perhaps at 100% usage for many seconds or minutes. Interrupts generated by peripherals may be processed slowly, or indefinitely delayed.

nevermind

oh so i was right

[sharp plover]

But threads can still speed up your program if it has lots of IO. One thread can be getting on with something else while another thread is waiting on IO.

Although now asyncio is generally a better approach to this.

[wanton pecan]

I was directed here from general. Anyone have experience with setting up a Docker configuration for Django, Nginx, Redis, Postgres, and Gunicorn? Specifically regarding securing the redis implementation and any security gotchas

[proper cairn]

Hey can someone help me figure something out?

[sharp plover]

I think this page has the information you're looking for: https://realpython.com/cpython-source-code-guide/#execution

If I understand you correctly, then yes, different threads have different callstacks.

[raven ridge]

Python threads are real OS threads, so yes, they each have their own call stack.

[sharp plover]

Hey 🙂 Go ahead and ask your question. People are less likely to reply to open-ended questions (as they don't know if they will be able to help effectively).

[crude turret]

ah alright

yeah that makes sense since python threads r just.. interpreters being launched, at least that's what it seems
https://cdn.discordapp.com/attachments/306885827189800960/784079937979809832/unknown.png

[radiant fulcrum]

Is this pythons threading type threads or MP style threading

[crude turret]

threading library yeah

[radiant fulcrum]

first time ive seen an OS actually list the os threads a process is using

[crude turret]

lol

what's an MP style thread?

[radiant fulcrum]

Multi Processing

what python has todo to be able to actually use threads/multiple cores

[crude turret]

ohh yeah

[silver edge]

No I don't think markdown has this feature yet.

[bitter abyss]

oh alright thanks for your help

[silver edge]

np

[radiant fulcrum]

probably because you wernt using it properly, CPU bound operations are pretty much always faster when using Multi processing when done correctly and threadings is pretty much always going to be slower (like massively slower)

[unkempt rock]

is there any reason why this wont work?

[grave jolt]

This is a channel for meta-discussions about Python itself. If you have a question, see #❓｜how-to-get-help. The issue is probably that the body of the function is not indented.

[unkempt rock]

ok sorry

[dawn hill]

GitHub MD has tables.

[unkempt rock]

ohhhhhhhhhhh so its function

[swift imp]

So yes, to put it more correctly. I overrided, metaclass __eq__ and as a result I also had to override __hash__ because when I tried to change the metaclass on a class that was being wrapped with dataclasses.dataclass the repl was erroring out because for some reason ipython was hashing classes

[grave jolt]

well, it should always hold that a == b => hash(a) == hash(b)

maybe it was using classes as dict keys

[swift imp]

from dataclasses import dataclass
class SpecialMeta(type):
     def __eq__(cls, other):
         if type(cls).__name__ == type(other).__name__:
             return True
         else:
             return False
@dataclass
class A(metaclass=SpecialMeta):
    a = 5

a = A()
a # raises unhashable type 'SpecialMeta'

ahh, apparently its going nothing to do with @dataclass errors out even without it

Oh it does

It hashes, the classes yes

I just thought it was odd

[raven ridge]

You're not being bitten by anything specific about metaclasses, then, but by the fact that objects that don't have an __eq__ are hashable, but types with an __eq__ are not unless they explicitly define __hash__

[swift imp]

Ah

What gets used on a class for default __hash__? Is it name? Because that's what I ended up using for the hash. I also never plan on hashing my class

[raven ridge]

It's its id()

[swift imp]

Ah

See I got a class factory of dataclass subclasses and I want to be able to compare the actual class and not instances. So I made my own __eq__.

[raven ridge]

!e ```py
class A:
pass
class B(A):
def eq(self, other):
return True
class C(B):
def hash(self):
return 42
print(hash(A()))
print(hash(C()))
print(hash(B()))

[fallen slate]

@raven ridge :x: Your eval job has completed with return code 1.

001 | 8742631872509
002 | 42
003 | Traceback (most recent call last):
004 |   File "<string>", line 11, in <module>
005 | TypeError: unhashable type: 'B'

[swift imp]

Ah

[raven ridge]

If your __eq__ considers the name, your __hash__ will need to as well.

[swift imp]

Well as of right now, my __hash__ only uses the name, but now I am thinking I should be using all things I check in __eq__

[raven ridge]

Yes, you need to.

The only rule that you must adhere to is that two objects that compare equal must have equal hash values

You can achieve that by returning a hardcoded value unconditionally, but it's more common (and much better if you expect the instances to commonly be hashed) to see __eq__ make a tuple of interesting values from each instance and compare the two tuples for equality, and for __hash__ to build the same tuple of instance variables and hash it.

[swift imp]

Thank you

[raven ridge]

Windows?

Multiprocessing on Windows is much less efficient than on Unix, because of the low level APIs that Windows gives. With Unix you can set up a bunch of data in the parent process and then fork a worker process that inherits them nearly for free. With Windows you must instead start a new empty process and then send data to the child process one object at a time by pickling it, which sucks.

You can maybe use multiprocess8ng.Manager instead, but that doesn't work for all types.

[grave jolt]

so there's no shared memory between processes on windows?,,

[raven ridge]

There's no sharing of an entire process address space like there is from fork on Unix

You can create a chunk of shared memory and write stuff into it explicitly, but that still requires copying it from the process's normal heap into the shared memory. And that can't be done on arbitrary Python objects. They all contain pointers to other objects, and those pointers can't be copied into shared memory, because the things they point to won't exist in the child process.

[grave jolt]

alright, this is getting a little bit off-topic... maybe #tools-and-devops or #async-and-concurrency is more appropriate

[raven ridge]

The best bet would be restructuring the flow so that instead of pixels being passed from parent to child, each child individually loads the image, and all that's passed from parent to child is the range of pixels for that child to work on.

You can from separate processes, unless it's somehow sharing state across process

No, there's not. Most things that aren't threadsafe are still process-safe. The slow part you're hitting is serializing data in one process and sending it to another. The way to speed that up is finding a way to restructure the program to avoid the copy, like doing work in multiple processes from the start so that there aren't intermediate states that need to be moved between processes.

I think specific solutions for that problem are definitely more in the scope of #async-and-concurrency

[swift imp]

Is there a list of special words that get highlighted by text editors the way TO DO, NOTE does?

[unkempt rock]

is a linked list advanced?

[raven ridge]

No, but #algos-and-data-structs is a good channel to discuss that

[grave jolt]

This channel is not for "advanced" questions. It's for meta-discussions about the Python language.

[crude arrow]

@swift imp https://www.python.org/dev/peps/pep-0350/#mnemonics

guys, is there a typing-related channel?

[grave jolt]

Not really. You can ask in a help channel.

[swift imp]

Anyone heard any good news about https://www.python.org/dev/peps/pep-0638/

[brave badger]

From what I can infer, 638 has a lot less division compared to 634~636

In terms of maintainability, I have to agree that 638 is more practical

Of course, it wouldn't be trivial to implement pattern matching with 638, but you'd have to consider that if they were to be implemented, they'd be syntactic macro code, not CPython code

[grave jolt]

I would absolutely love to see do notation in Python

[candid stump]

is it possible to call gRPC api's with Postman?

[brave badger]

do-it-yourself

Syntactic macros would cause more chaos in #esoteric-python

[radiant garden]

#esoteric-python sends back a message:

from theoretical_do_while import do, _while
with do:
  ...
_while(condition)

theoretical implementation based on the non-theoretical goto implementation that jumps stack frames

[sacred yew]

I think they were referring to haskell do

[radiant garden]

mmm monads

[grave jolt]

yep

[brave badger]

https://twitter.com/raymondh/status/1334669384170627073?s=20

Raymond Hettinger (@raymondh)

#Python factlet: The set_name() method isn't part of the descriptor protocol. It will work with any class:

class A:
def set_name(self, owner, name):
print(f'{self} was named {name} by {owner}')

class B:
x = A()
y = A()

Likes

121

Twitter · Dec 4, 2020, 1:22 AM

Huh

Interesting

[wide shuttle]

Yeah, it's just listed under the descriptor section, but there's nothing about it only running if the other descriptor protocol methods are present

[torn sedge]

!subscribe

[deft pagoda]

hmm, that's something I will try to abuse

[wide shuttle]

I ran into something (entirely valid and explainable) the other day that I found quite funny.

Basically, I defined an abstract base class to be used as the base for dataclasses. No problem so far, but the method I wanted to declare abstract was __init__. This leads to the interesting situation of having a working, concrete in the class (after the dataclass decorator has done its work), but __init__ still listed as an abstract method in the dataclass's __abstractmethods__ frozenset and, thus, instantiation leads to an exception.

!e

import abc
import dataclasses
from argparse import Namespace


# --------------------------------------------------------
# Abstract Base Class
class MyABC(abc.ABC):
    @abc.abstractmethod
    def __init__(self, *args, **kwargs):
        raise NotImplementedError


print(
    "Is MyABC.__init__ an abstract method:",
    hasattr(MyABC.__init__, "__isabstractmethod__")
)


# --------------------------------------------------------
# Concrete Class (sorta)
@dataclasses.dataclass(frozen=True)
class MyConcreteClass(MyABC):
    attribute: str


print(
    "Is MyConcreteClass.__init__ an abstract method:",
    hasattr(MyConcreteClass.__init__, "__isabstractmethod__")
)

# The __init__ works
dummy_object = Namespace()
MyConcreteClass.__init__(dummy_object, attribute="value for attribute")
print("Value of dummy_object.attribute:", dummy_object.attribute)

# Show that the `__init__` method is still listed as abstract methods of the class:
print("Current abstract methods listed for MyConcreteClass:", MyConcreteClass.__abstractmethods__)

print("-----------------------------------------------------")
# Actual instantiation fails
instance = MyConcreteClass(attribute="value for attribute")

[fallen slate]

@wide shuttle :x: Your eval job has completed with return code 1.

001 | Is MyABC.__init__ an abstract method: True
002 | Is MyConcreteClass.__init__ an abstract method: False
003 | Value of dummy_object.attribute: value for attribute
004 | Current abstract methods listed for MyConcreteClass: frozenset({'__init__'})
005 | -----------------------------------------------------
006 | Traceback (most recent call last):
007 |   File "<string>", line 47, in <module>
008 | TypeError: Can't instantiate abstract class MyConcreteClass with abstract method __init__

[wide shuttle]

This is obviously because dataclass monkeypatches a (locally defined) function onto the __init__ attribute of the actual class object after it's been created

You can avoid this by monkeypatching __abstractmethods__ yourself, but that obviously brings you even further into the distasteful territory of hacks

!e

import abc
import dataclasses

# --------------------------------------------------------
# Abstract Base Class
class MyABC(abc.ABC):
    @abc.abstractmethod
    def __init__(self, *args, **kwargs):
        raise NotImplementedError


# --------------------------------------------------------
# Concrete Class (sorta)
@dataclasses.dataclass(frozen=True)
class MyConcreteClass(MyABC):
    attribute: str


MyConcreteClass.__abstractmethods__ = frozenset()
instance = MyConcreteClass(attribute="value for attribute")
print(instance.attribute)

[fallen slate]

@wide shuttle :white_check_mark: Your eval job has completed with return code 0.

value for attribute

[wide shuttle]

Apparently the __abstracmethods__ attribute is set after __init_subclass__ runs or that's what this makes me think:

import abc
import dataclasses


# --------------------------------------------------------
# Abstract Base Class
class MyABC(abc.ABC):
    @abc.abstractmethod
    def __init__(self, *args, **kwargs):
        raise NotImplementedError

    @classmethod
    def __init_subclass__(cls):
        print("Has attribute before patching:", hasattr(cls, "__abstractmethods__"))
        cls.__abstractmethods__ = frozenset()
        print("Has attribute after patching:", hasattr(cls, "__abstractmethods__"))


# --------------------------------------------------------
# Concrete Class (sorta)
@dataclasses.dataclass(frozen=True)
class MyConcreteClass(MyABC):
    attribute: str


print("Attribute after being fully initialized:", MyConcreteClass.__abstractmethods__)

Has attribute before patching: False
Has attribute after patching: True
Attribute after being fully initialized: frozenset({'__init__'})

Could probably do this with a decorator, though

[deft pagoda]

does cluegen have this problem

it would see the methods defined i think, even if they're just descriptors?

[wide shuttle]

cluegen uses subclassing, right?

I think that could work, depending on who it's implemented

It uses __init_subclass__, as I suspected, which runs before the frozenset for __abstractmethods__ is created

So, it should work

Oh, not even that. It does use it, but it uses some other magic and the methods are always already defined

[undone hare]

I wonder something, does someone know why most range in Python (slice, range, ...) has an inclusive start but an exclusive end?

[wide shuttle]

There are a few reasons for that; at least, according to the tutorial in the docs

[deft pagoda]

i can link the famous article about it

https://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/EWD831.html

[wide shuttle]

Things like:

string[:3]  # first three, 0, 1 2
string[3:]  # 3+
# so
string == string[:3] + string[3:] 

# range
range(5)  # range with 5 elements, 0..4
# so
for i in range(5):
    # will loop 5 times

[deft pagoda]

found it

[wide shuttle]

Is that the 0-based explanation from Dijkstra?

[deft pagoda]

yeah, there's also a note by gvr somewhere, i can find that too

[magic python]

@grave jolt sorry to ping but i'm curious about the do comment, I tried to skim https://en.wikibooks.org/wiki/Haskell/do_notation but didn't really know what it's about. In the context of python, what would do provide?

[deft pagoda]

https://python-history.blogspot.com/2013/10/why-python-uses-0-based-indexing.html that took a long time to find

[magic python]

ABC, one of Python's predecessors

does this have anything to do with the ABC classes or something? (i never use them - but seem to recall seeing them)

[wide shuttle]

No, it's a language that was developed with teaching in mind

Guido worked on developing ABC at the Centre/National Research Institute for Mathematics and Informatica (theoretical CS) in Amsterdam

[magic python]

ah ok, interesting, cheers

[undone hare]

I always assumed zero-basing was because of C

[lofty rover]

Hey guys, in regards to running ML on external kernels, does anyone have some tips or best practices?

I'm trying to setup an environment where I can colab with people on the same kernel, keep it up and running so resources dont have to be reloaded

I was thinking of just setting up an ec2 instance on AWS to run it, but I'm not sure if that would be efficient. Perhaps something like google colab would be better?

Most free things have their kernel shut down after 6/9/12 hours

[deft pagoda]

this room is for discussion of the python programming language --- maybe try #data-science-and-ml

[grave jolt]

It's about writing a chain of flat_maps (aka then/>>=):

def f(c):
    return foo(1, 2, c).then(lambda x: bar(42, x).then(lambda y: fizz.then(lambda _: buzz(x, y, 5))))

as this:

def f(c):
    return do!:
        x := foo(1, 2, c)
        y := bar(42, x)
        fizz
        buzz(x, y, 5)

and hopefully that'd be possible with macros

[magic python]

@grave jolt oh right... cool... don't fully get it but it looks a lot cleaner, kinda like piping or something, which is good with me

[grave jolt]

if you have a map function: map: (A -> B) -> (M[A] -> M[B}) and a flatten function: flatten: M[M[A]] -> M[A], then you can create a flat_map function: flat_map: M[A] -> (a -> M[B]) -> M[B]. Example:

flat_map([10, 20, 30], lambda x: [str(x**2), str(x**2 + 1)]) == ['100', '101', '400', '401', '900', '901']

[magic python]

will this work with mypy?

i recently bumped into something that meant i couldn't use collapse or something in my code, which seems related to the flatten part

[grave jolt]

well, you can't really easily overload free-standing functions in Python

[magic python]

(collapse from moreitertools)

I'm not sure what the most natural way to do that flat_map would usually be

something like this seems too messy:

l = [10, 20, 30]
(
    [
        num
        for lst in list(zip([x ** 2 for x in l], [x ** 2 + 1 for x in l]))
        for num in lst
    ]
)

wrapping zip in list is not needed there... but whatever 🙂

[unkempt rock]

where can i ask question?

[worldly knoll]

Anywhere you want

@unkempt rock

[unkempt rock]

ok

well whenever i use a script without an error my computer says that blender isn't responding why

[magic python]

@worldly knoll no - in the right channel, that's the point of channels

[worldly knoll]

@magic python , im sorry

[sage scroll]

Anyone up for review my project and can give suggestions?

[grave jolt]

Hi, that's probably more on topic for #software-architecture

(if you're asking about the code)

othewise, #python-discussion or ot-channels

[sage scroll]

Alright

[smoky turret]

there ever been a pep proposing some form of is_even()/is_odd() methods on ints?

I havent found any

[feral cedar]

good thing you can import them

https://pypi.org/project/is-even/

[spark magnet]

@smoky turret it's easy enough to do it yourself

[smoky turret]

obviously implementing them is trivial

[feral cedar]

then you can implement them lol

[smoky turret]

but Ive never been a fan of how it looks in code

[feral cedar]

you can always use & 1 instead of % 2

[smoky turret]

if num % 2 != 0 always struck me as really ugly

[severe lichen]

Never seen that. Probably because that's the same as num % 2 == 0. And operator module probably has modulo and equals if you need it in function form (for map, for example)

[spark magnet]

define a function then

[feral cedar]

class Integer++ extends int

[smoky turret]

are people here really so against a form of is_even() method as a part of the int class?

[spark magnet]

@smoky turret we are explaining how the idea would be met by the core devs.

[feral cedar]

i'm not against it, but i don't see why it's necessarily. everyone already recognizes num % 2 as checking for parity

[peak spoke]

Feels like bloat on the type that already has good solutions

[smoky turret]

I see this kind of argument a lot, and Ive never been a fan. Its only super obvious when youve seen and used it a lot, but it still takes a second to parse, and during debugging you still have to stop to make sure you got == or != correct

[feral cedar]

but anyone who is debugging or reading large amounts of code has seen and used it enough to get it right away

[severe lichen]

Also, consider this. How often do you need to check if number is even outside of programming exercises?

[smoky turret]

and I know on personal projects ive done it at least a few times

[severe lichen]

Hm, weird, but you probably has more experience, so i retract my statement

[smoky turret]

evenness seems reasonable enough for a integer class

and no one can deny that if num.is_even(): just looks and reads much better

[severe lichen]

There's always little things you want to do, and probably noone has time for that.
For example, I'd like to have mutable string built-in that works similarily to list of 1-length string, but has string methods.

But if I do a PEP, people would just say "make it yourself".

[smoky turret]

its strikes me as fundamental enough to deserve its place in the language itself

its not like we still arent changing the basic types either

we just got those two new prefix methods on strings

https://www.python.org/dev/peps/pep-0616/

[spark magnet]

that's because .strip was confusing people

it was a common thing that people were getting wrong. even and odd are not like that.

[severe lichen]

That reminds me... Is there a reason why you can't define extension methods?

Basically, allow to add methods to already-made types. I think you can already do that with monkeypatching, but it's seen as very bad practice.

[spark magnet]

it's not allowed because it's seen as bad practice.

[pliant tusk]

@severe lichen if you really want to extend builtin types, this module i made is currently the most stable (afaik): https://pypi.org/project/fishhook/

[severe lichen]

Thanks! Also, I've heard of forbiddenfruit.

[pliant tusk]

mine is different from forbiddenfruit because exceptions work in my patches

[severe lichen]

Unfortunately, messing with built-ins is probably not worth it for simply postfix notation vs prefix.

And yeah, I hate stuff line len(list) instead of list.len()

[feral cedar]

at least it's consistent

[raven ridge]

From the point of view of the language maintainers, orthogonality is an important point. They don't want there to be a different ways to do the same thing, but a handful of sufficiently powerful primitive so that other things can be composed.

somestr.removeprefix("foobar") is equivalent to somestr[len("foobar"):] if somestr.startswith("foobar") else somestr - which is an awful lot to type.
some_int.is_odd() is equivalent to bool(some_int % 2)- the latter is more general without being onerous

[pliant tusk]

@severe lichen ```py
from fishhook import hook_cls

@hook_cls(list)
class list_patch:
def len(self):
return len(self)
def map(self, func):
return map(func, self)

adds .len and .map to lists

(also if this doesnt fit in #internals-and-peps ill move it to #esoteric-python )

[raven ridge]

in addition to the question about whether it would be hard to implement yourself, it's also worth asking what makes evenness and oddness special. Why have checks for evenness and oddness as dedicated methods of integers, instead of methods for negativeness or logarithm or twos complement or factorial?

[vale flax]

i hate making map a method as opposed to a function

i would rather lobby for a pep where |> becomes a standard infix operator

[grave jolt]

Universal call syntax (i.e. f(a, b) <=> a.f(b) would be nice, although not possible in Python

[pliant tusk]

it is possible with some monkey patching

[smoky turret]

I wasnt suggesting some fancy new calling syntax, mearly the addition of a new int method

[grave jolt]

I understand

[pliant tusk]

not with vanilla python tho

[smoky turret]

the iter2 module implements that kind of chaining with a wrapper with a custom getattr https://pypi.org/project/iter2/

I made my own toy version awhile ago bc I thought it was cool

[grave jolt]

But, as godlygeek said, what's special about evenness?

Adding more and more methods to a type really bloats it. It also makes calls to built-in functions look different from your own functions, so you can't really write your own logic, e.g. if you want to see if a number is divisible by 3, we're back to square one

[swift imp]

Why is parity check useful? Is that a serious question?

[raven ridge]

Why is it more useful than a negativeness check?

[smoky turret]

what aspects of any type are "useful" enough to warrant inclusion is a judgement call. For me it seems clearly good enough

[swift imp]

I mean it probably shouldn't be a method because its trivial to do with x%2

but like why is a parity check useful?

Common, its a huge operation, so huge theres dedicated logic for it in CPU instructions

[smoky turret]

as I explained above, I dont find x%2 nearly as readable as something like if x < 0 which is why I like an is_even(), and not so much a is_negative()

[swift imp]

its an incredibly important application thats used in a lot of fields

[grave jolt]

Not sure if it's a .NET thing or a Pascal thing, but Pascal ABC .NET has a thing called "Extension methods", where you can extend existing types with methods, but don't actually "add" them to a class. This allows for extending types and maintaining the same syntax.

procedure MyPrint(Self: integer);
begin
  writeln(Self)
end;

procedure MyPrintEx(Self: integer); extensionmethod;
begin
  writeln(Self)
end;

begin
  MyPrint(1);
  1.MyPrintEx;
end.

[severe lichen]

C# too has it, so it's .Net

[spark magnet]

@grave jolt this is essentially encouraging mutable globals

[grave jolt]

huh?

It's not actually mutating a class or anything, it's just another syntax variant for function call.

[spark magnet]

@grave jolt but that new method is available to all users of the class? or it isn't?

[grave jolt]

No, it's just available in a single module, like a free-standing function.

[spark magnet]

I see.

that's better than ruby-style monkeypatching

[raven ridge]

I guess I'm not sympathetic to this argument because I find if x % 2 == 0 and if x < 0 approximately equally easy to parse.

[feral cedar]

i think that's the main point against it

[spark magnet]

@smoky turret there will always be things that seem obviously good that are missing or wrong in Python. any() should return the actual True-ish value it found. It was considered, and rejected for what i consider to be silly reasons. Oh well. 😦

[raven ridge]

I didn't realize it doesn't. That's sad...

[swift imp]

Uh

[pliant tusk]

@spark magnet that makes sense to me for any() as it works on any collection of values (and i believe it does not specify in what order) . If the user expected it to return a specific first truthy value then it could cause unexpected behavior

[swift imp]

next((thing for thing in iterable if conditon,) False) Theres you're any()

[grave jolt]

ew

[swift imp]

ew?

[grave jolt]

Well, maybe you could make a first or coalesce function

writing that whole thing out might not be ideal

[swift imp]

next((thing for thing in iterable if conditon,) False) will return the first thing found to be True now if you wanted literally all of the things that are True, I'd argue that muddies with all(). Like if Ned was saying that any() should return everything that evaluates True, I think that conflicts with all() and doesn't make sense.

[grave jolt]

😦
Also, some things are the way they are because Python has undergone some evolution and has to be backwards compatible. For example, lots of weird things like id or eval in the global scope; or bool being a subclass of int.

[raven ridge]

that one bites people occasionally, heh

[raven ridge]

hm, yeah, but - it wouldn't be hard to add an itertools.any that returns the first truthy value, and then deprecate the builtin any with bool(itertools.any(...)) as its documented replacement. Maybe for 4.0 😄

[swift imp]

Why deprecate any

[raven ridge]

two birds with one stone, getting something rarely used out of the builtin namespace, and making it behave like a or b or c or d does.

[swift imp]

Like I don't get it. It short circuits as soon as the condition is True, whats so bad about that

[raven ridge]

so would a or b or c or d

[swift imp]

But if I have to logical or N things?

[raven ridge]

sure, my proposed itertools.any would take exactly that.

[swift imp]

Thats weird

[raven ridge]

and it would return the first truthy one of them, or the last one if none are truthy.

[swift imp]

What

Seems like it should be different itertools function or builtin rather than deprecating a really useful (imo) builtin

[severe lichen]

I feel #esoteric-python is leaking hard

BTW, add itertools.top that works like max() for several values.

[raven ridge]

well, if you want to keep the builtin, you could redefine it to return the first truthy element, instead of returning True if any elements are truthy. The only disadvantage to that is that it makes for a harder migration.

[swift imp]

I literally don't understand why you want it to return the first truthy object or the last if none are truthy

[raven ridge]

replacing any with an itertools.any that works that way would be easier from a backwards (and forwards) compatibility PoV

well, for the same reason as or returns the first truthy object. It's less work, and sometimes it's helpful to know which of the two things were truthy.

[swift imp]

just use next

[raven ridge]

I mean, I'd just use a for loop instead, but it would be nice if I could use any instead

[swift imp]

if both operands eval False then or returns False. If any of the operands are True, it returns the first one True. That is literally what next((thing for thing in iterable if thing), False) does

[raven ridge]

!e ```python
print("" or [])

[fallen slate]

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

[]

[swift imp]

ok

I was wrong

Id argue thats bad form

[raven ridge]

it's literally how the operator is defined to behave; I can't see how depending on it could be considered bad form.

[swift imp]

Bc its confusing as hell thats why, then again the whole thing for truthyness is

[raven ridge]

And it's a relatively common idiom for default values...

if (input("Continue? [Yn]") or "y") == "y":
    do_thing()

[swift imp]

I mean look, if you're dealing with truthyness as you are here. Then this does exactly what you want. next((thing for thing in list_of_things if thing) , list_of_things[-1])

[raven ridge]

that only works on sequences, not arbitrary iterables

But yeah, look - it's not that it's difficult to define an any that works the way I want, it's that it's odd that the builtin any doesn't work the way I want.

It's strange and surprising that it applies a bool() to the result when nothing else in the language behaves that way.

[swift imp]

You can use the or pattern on arbitrary iterables?

Like doesnt that fall back to using __len__, how does it work with generators?

[raven ridge]

you could do it with operator.or_ and functools.reduce() , for instance.

or just:

def any(iterable):
    for val in iterable:
        if val:
            return val
    return val

it's weird to me that the current any finds a truthy value and then throws it away instead of returning it. It does more work to give less information.

[swift imp]

I see your point, but its like

Idk. I see any and all and they follow suit of how Matlab does it (lol)

¯_(ツ)_/¯

good talk, thanks for putting up with my gripes

[raven ridge]

it's all wishful thinking anyway - maybe one day for a 4.0, since it's a relatively minor backwards incompatible change that might be justifiable, but it's needed rarely enough and there's so little appetite for breaking changes that it's unlikely to happen regardless.

[swift imp]

I almost feel like next should have a better default value argument

[severe lichen]

BTW, what do you feel about introducing dependencies solely because of syntaxic sugar? I'm talking mainly about more-itertools module.

[swift imp]

Like maybe take a function that can act on the iterator

[raven ridge]

hm - why? If the default is used it's because the iterable has been exhausted, and there's nothing that a function can possibly do with an exhausted iterator

[swift imp]

That or change or to return False if nothing is truth-y for consistency.

I see your point is all, and am trying to think about it in terms of next

like this is just silly '' and 'a' that it returns '' imo but the or application you present is more useful. So like would you want all to changed too?

[raven ridge]

yep, I would.

Note that a or b is basically a if a else b as things stand today, and a and b is a if not a else b

[swift imp]

I think thats a major fuck up of their truthy system. Its so god damn confusing

[raven ridge]

javascript behaves the same way. I think a fair number of languages do.

[severe lichen]

I wonder if the designer of that feature (Guido himself?) thought "It would be faster that way", "People might want to use the truthy value" or "#FIXME : Quick temporary hack"

[swift imp]

Whats the reasoning of falling back to len? I guess thats NOT that bad, but the fact that and and or don't return a bool is whats weird imo.

[raven ridge]

I'm not sure what you mean by "falling back to len"

[raven ridge]

I'm pretty sure that Python inherited this feature from ABC

[swift imp]

if __bool__ isn't implemented on an object, then __len__ is used

Hence empty iterables evaluate False because they have zero length

[raven ridge]

not all iterables have __len__ ...

[swift imp]

I think and then if no __len__ then the object is True

[hexed island]

@swift imp Why are you so angry?

[raven ridge]

but you're right, that's a problem with my proposed implementation of any above

This:

def any(iterable):
    for val in iterable:
        if val:
            return val
    return val

would raise an UnboundLocalError if the loop executed 0 times.

[swift imp]

So a generator doesn't have len but it implements bool

[raven ridge]

So:

def any(iterable):
    for val in iterable:
        if val:
            return val
    try:
        return val
    except UnboundLocalError:
        return False

😄

generators don't implement bool

!e ```py
print(bool(x for x in []))

[fallen slate]

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

True

[raven ridge]

that's True despite being an empty (exhausted) generator.

[swift imp]

Then how does it evaluate True given that len doesn't work on them

[raven ridge]

!e ```py
print(bool(object()))

[fallen slate]

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

True

[hexed island]

...

[swift imp]

By default, an object is considered true unless its class defines either a bool() method that returns False or a len() method that returns zero, when called with the object.

[raven ridge]

Everything's truthy except for things with a __len__ that returns 0, a __bool__ that returns False, or an empty string, list, tuple, bytes, ..., etc

[swift imp]

Yeah man idk how to feel

Either change any and all or change or and and, I'm leaning towards the latter

Id love to read that PEP, if you have a link

[raven ridge]

heh

[feral cedar]

just turn any into eval(" or ".join(args))

[raven ridge]

blinking man meme

[severe lichen]

Nice

[mint forge]

Can u not have an instance of a metaclass?

[raven ridge]

A class is an instance of a metaclass.

[mint forge]

oh

[raven ridge]

By definition. The type of a class is its metaclass.

[mint forge]

is there a way to make an instance of metaclass without doing
class Something(metaclass=Somethingelse)

[raven ridge]

Sure. Call it.

[mint forge]

but it will give __new__ takes 3 arguments none were given

[raven ridge]

Right - so you can provide them

[mint forge]

what are those 3 arguments?

[raven ridge]

you're inheriting from type, so they're the 3 arguments that type takes - the name of the class, a tuple of base classes that it inherits from, and a class dict.

!e ```py
cls = type("Foo", (), {})
print(cls)

[fallen slate]

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

<class '__main__.Foo'>

[mint forge]

ohhhhh

i see thx a lot

[mint forge]

so is cls is a metaclass?

[raven ridge]

no, cls is a class. type is a metaclass, because it's the type of the cls class.

and the type of a class is its metaclass.

[mint forge]

but the type of a class whose type is type isn't that a metaclass?
and when we do print(type(cls)) it prints <class 'type'>

[raven ridge]

but the type of a class whose type is type isn't that a metaclass?
No. The type of a class is the metaclass of that class. The metaclass of Foo is type.

the type of all classes is either type or a subclass of type, generally.

[mint forge]

oh

[raven ridge]

because a class "is a" type, in OOP terms. Classes inherit from type because every class is-a type.

!e ```py
class Foo: pass
print(type(Foo))

[fallen slate]

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

<class 'type'>

[mint forge]

oh

[raven ridge]

(in fact, class Foo: pass and Foo = type("Foo", (), {}) are two different ways of writing the exact same thing)

[mint forge]

right, so how would we define a metaclass?.

[raven ridge]

a metaclass is a class. The thing that makes it a metaclass is making it the type of a class, by setting it as that class's metaclass.

a metaclass is a class that happens to be the type of another class.

[mint forge]

how is it a type of another class

[feral cedar]

just like 6 is an int, a class is a metaclass using oop terms

[raven ridge]

because that other class is defined using metaclass=YourClass

up until you do that, it's just a regular old class. The thing that makes it a metaclass is using it as the metaclass of another class.

[mint forge]

i think i kinda get what u are saying

[raven ridge]

a metaclass is a class that is the metaclass of a class. If it's not the metaclass of a class, it's not a metaclass.

[mint forge]

it becomes a metaclass class A(metaclass=Something)
because A did metaclass=Something?.
also if every class is a metaclass then why do we need to do

class A(type):

[raven ridge]

!e ```py
class MyMetaClass(type):
pass

MyMetaClass isn't a metaclass, because there's no

class with MyMetaClass as its type

class Foo(metaclass=MyMetaClass):
pass

print(type(Foo))

Now there is one, so it's a metaclass!

[fallen slate]

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

<class '__main__.MyMetaClass'>

[mint forge]

yes i get that part

but if every class is a metaclass then why do we need to do

class something(type)

[raven ridge]

to inherit from type, so that your class does all the things that all other types do.

[mint forge]

but aren't all classes a subclass of type?

[raven ridge]

classes are instances of whatever their metaclass is. If your metaclass isn't a type, then classes made with that metaclass aren't types either.

[mint forge]

hmmmmm,
huh

[raven ridge]

it's not really useful to make a metaclass that's not a subclass of type, because if you did then classes using your metaclass wouldn't be types, and if they aren't types then what would they be?

[mint forge]

lets see what they would

[raven ridge]

but you still need to list (type) every time because metaclasses are just regular classes, and you need to declare your base classes for them just like you would for any other class.

[mint forge]

wdym "base classes"

[mint forge]

#bot-commands message
error

[raven ridge]

that's because the class definition tries to construct the type, by passing it its 3 arguments

[mint forge]

mhm

[raven ridge]

!e ```py
class Foo(metaclass=print): pass

[fallen slate]

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

Foo () {'__module__': '__main__', '__qualname__': 'Foo'}

[mint forge]

wait print is a class

[raven ridge]

no.

[mint forge]

then how were u able to do metaclass=print

[raven ridge]

what a class statement does is call whatever type is listed as metaclass (or type if no metaclass= is given) with 3 arguments: the name of the class, the base classes, and the class dictionary.

[mint forge]

hmm

wdym by the "base class"

[raven ridge]

the classes it inherits from.

[mint forge]

so if it doesn't inherit from anything then its base calss is type right?

[raven ridge]

no, it's object

[mint forge]

oh

[real cave]

I have a question about why Python allows one but not the second example:

1. I can access the chars contained in a string using result = str[index] with the expected result containing the char pointed at by index.

2. However, I cannot then turn around and attempt to write to the string with:
   Str[index] = result

The error reads "str class does not accept assignment " and I want to understand WHY a str is not treated as a list of chars. New to Python and curious. Thanks for your thoughts

[mint forge]

because strings are immutable, they cannot be changed

[real cave]

Yet I can assemble them one char at atime.... and who decides what is immutable?

[feral cedar]

you don't modify the string, you created a new one

[unkempt rock]

hi,i am new to python
tell me how many spaces in while loop for everything coz i keep getting error

why do i keep getting this

[mint forge]

@unkempt rock this is not the channel for that, See #❓｜how-to-get-help

[unkempt rock]

oh ok sorry

[pliant tusk]

@real cave strings in python cannot be modified in place. you can add to them, which makes a copy with whatever being added concatenated to it

x = 'a'
print(id(x))
x += 'b'
print(id(x))

if you run that youll see that the id changes

[real cave]

So, each time I append a char to the output str a new str is created? Sounds messy and memory eating!

[feral cedar]

yep, that's why you use join

[severe lichen]

<= this

[raven ridge]

@mint forge the word "metaclass" is a lot like the word "weapon". What makes something a metaclass is how it's used, not something intrinsic to it. A thing is a weapon if you use it to threaten or attack people. A thing is a metaclass if you create a class whose type is that thing. Threatening or attacking people with an object doesn't change what it is, it just gives us a new word to describe it, "weapon". Setting a class as the metaclass= of another class doesn't change what it is, it just gives us a new word to describe it, "metaclass".

[deft pagoda]

only people?

[severe lichen]

But immutable strings allow you to do stuff like {"black": 1, "white": 0}, so I'll take that.

[raven ridge]

I can use a shoe as a weapon, but not all shoes are weapons, and a shoe doesn't become a weapon until I use it as a weapon. A class doesn't become a metaclass until I use it as a metaclass.

[mint forge]

i kinda understand let me play around with it a bit more

[severe lichen]

I think it's about giving a gun to a monkey, AKA a bad idea.

[real cave]

@Oouja what does that code phrase do?

[severe lichen]

Question about metaclasses - what are problems that are easier to solve with metaclasses than with decorators.

[pliant tusk]

@real cave {'a':1, 'b':2} is a dictionary

[sacred yew]

enum and dataclassrs are both implemented using meta iirc

[pliant tusk]

you can index them by string like dct['a']

[severe lichen]

Ah, I tried to joke, but i misread the previous sentence making the joke to miss its target.

[raven ridge]

The purpose of a metaclass is to change how the class behaves when you call methods on it. You can't in general do that with decorators - you can patch attributes onto the class instance after the fact, but not everything that you might want to customize can be done that way. For instance, a metaclass can change the name of a class or the base classes of a class, and therefore its MRO. A class decorator cannot do that (though it could create a new subclass that wraps the original class, having a different name or a different set of a base classes)

[severe lichen]

Thanks! But I was talking more about concrete examples.
Enums are a nice example, where class definition instead creates a number of items instead of using it as attributes, like normal class. I'm not sure I'd be able to replicate the functionality with a class decorator.

[raven ridge]

one useful thing about metaclasses is that they're inherited. With a decorator, you'd need to decorate every class that you want to apply the behavior to. With a metaclass, you need to only add metaclass=Foo to one of them, and and it's automatically applied to all the subclasses.

this lets you hide some of the magic from your users.

[severe lichen]

Hm, didn't think about it. Indeed, that's a major selling point and can prevent a lot of bugs when people forget to decorate stuff.

[raven ridge]

!e ```py
from string import Template
class UnderscoreTemplate(Template):
delimiter = "_"
template = UnderscoreTemplate("_greeting, _place!")
print(template.substitute(greeting="Hello", place="world"))
print(type(UnderscoreTemplate))

[fallen slate]

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

001 | Hello, world!
002 | <class 'type'>

[raven ridge]

hm, that used to be a metaclass - is it not anymore? 😄

ah, no - it uses __init_subclass__ now, instead.

but back in the day, https://github.com/python/cpython/blob/2.7/Lib/string.py#L102

[grave jolt]

lmao

I'm stealing this

[cloud crypt]

LOL

I can't stop lmao

this is great

[spark magnet]

@pliant tusk any() operates in order, same as any other thing that iterates a collection.

[deft pagoda]

sounds like a challenge to me

def random_iterator(iterable):
    for _ in iterable:
        yield choice(iterable)

only works for sequences though 😦

[spark magnet]

each collection may have a "random" or "unpredictable" order, but any() doesn't introduce any uncertainty about the order (is what I meant)

[finite jasper]

idk if i should ask this here but not many beginners use vim
how to auto format python code with neovim?

[deft pagoda]

#tools-and-devops

[radiant garden]

Haha that's even better than @print decorators

[wide shuttle]

Magic! I like it.

[swift imp]

I'd like to ask what people would do with __set_name__ outside of descriptors? Like

class Foo:

  def __set_name__(self, owner, value):
      pass

class Bar:
  foo = Foo()

[undone hare]

I believe we use it for our package async-redis-cache, to have the name of the cache, I don't think it is a bad idea

[swift imp]

Interesting, so you guys use it like a registry of sorts?

[unkempt rock]

is the idea of the operator .= nice?

like str.=split()

instead of str=str.split()

[feral cedar]

i don't think it's used that often to warrant that

[flat gazelle]

it also seems pretty difficult to give consistent semantics to

str .= name[:]

[sacred yew]

also it looks confusing

it looks like a normal function call, but its actually a method call

[unkempt rock]

wait what .= is a thing?

wait idea. nvm

[fresh cargo]

But if you squeeze your eyes really hard ,= is a thing

[neat cypress]

Fun challenge: explain why this snippet segfaults the cpython interpreter (requires cpython 3.8+)

(lambda:0).__class__(eval(f"lambda:{list(range(99999))} if 0 else -1").__code__.replace(co_consts=()),{})()

[grave jolt]

#esoteric-python

[neat cypress]

ah, nice

[burnt glacier]

Is something like this possible in tkinter?

if button_1.click

    if button_1.click

[cloud crypt]

class Data(Generic[T], metaclass=DataMeta, name="Data"):
    ...``` any idea how to do something like this on Python 3.6? I get ```python
TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases.```

DataMeta just inherits from type and takes some more arguments in __new__()

[runic musk]

try using with GenericMeta and type in DataMeta like this DataMeta(GenericMeta, type)

[viscid yoke]

I have a list of coordinates at (time 0 , time 1 etc...) and I want an animation of it
[x1(t0),y1(t0),x1(t1),y1(t1)....... x2(t0),y2(t0)]
x1(0) for first object at time 0 and x2(t0) for second object at time 0 etc
all of this is to create an animation of multiple balls moving
I don't know how to do it , what to use

[sacred yew]

@viscid yoke this is not a help channel, see #❓｜how-to-get-help

[unkempt rock]

Is this __init_subclass__ supposed to be like this?

>>> class b:
    def __init_subclass__(cls,**_dict):
        for elm in _dict:
            setattr(cls,elm,_dict[elm])

            
>>> class a(b,c='a'):
    pass

>>> a.c
'a'

[raven ridge]

I don't understand the question

[unkempt rock]

if i do class a(c='a'): it says TypeError: __init_subclass__() takes no keyword arguments

so am i supposed to make a base class and set __init_subclass__ to it?

[radiant garden]

how would you be able to call __init_subclass__ otherwise?

[unkempt rock]

ah

makes sense

thanks

__init_subclass__ was just made just for avoiding the usage of metaclasses right?

[unkempt rock]

What are you doing here?

[potent grove]

any good link to have an overview of cpython internal for the purpose of performance optimization ? preferably not the full source code 😄

but more higher level, like the GIL and stuff

[undone hare]

I believe RealPython has a book about the internals of CPython, but I'm not sure how good it is for performance optimization

[undone hare]

Sounds like something you’d want to ask in off-topic @bronze sleet

[bronze sleet]

ops, I thought Im in off topic

sorry

[undone hare]

No worries

[graceful kayak]

Is there a package that incrementally removes pip's cache? Like right now you can select which directories to remove in the cache, but I'd like to make this a cron job of some sorts. If not, is there a reason why it hasn't been implemented?

[undone hare]

I believe there's a command to clear it, although it isn't the right channel for that

[graceful kayak]

Probably to clear the entire cache, but that wouldn't be beneficial if you already have the latest version of a package but also still want to keep old versions that are still continuously used... and err which channel? #tools-and-devops or #unix ? confused

[mighty viper]

hello guys i had made a trained model for speech recognition and i test it but i test it on 1 sec wav file ,i want to make real time prediction script, can anyone help me

[grave jolt]

This is not a help channel, this is a channel for meta-discussions about the Python language. See #❓｜how-to-get-help or #data-science-and-ml

[mighty viper]

k thx

[sacred yew]

@graceful kayak #tools-and-devops

[patent mica]

guys ?

[undone hare]

Yes?

[patent mica]

so

i want to know what are the differences between TabError and IndentationError

like someone got IndentationError instead of TabError and it confused me

[undone hare]

!d IndentationError

[fallen slate]

`IndentationError`

exception IndentationError```
Base class for syntax errors related to incorrect indentation. This is a subclass of [`SyntaxError`](#SyntaxError "SyntaxError").

[undone hare]

!d TabError

[fallen slate]

`TabError`

exception TabError```
Raised when indentation contains an inconsistent use of tabs and spaces. This is a subclass of [`IndentationError`](#IndentationError "IndentationError").

[patent mica]

oh subclass

ok

[undone hare]

Yup

[patent mica]

but why do they appear in different moments when im almost doing the same mistake

[undone hare]

IdentationError is more general

[patent mica]

can i !eval here to show u ?

[patent mica]

oh so u get one of those randomly

[undone hare]

Let's go to an help channel, shall we

Can you ping me in one?

[patent mica]

if so, python is awesome

ok

[wide shuttle]

@patent mica You don't get one randomly, but it's based on the type of indentation error Python encounters while parsing your code. This specific error is raised specifically when you accidentally use both space characters and tab characters to indent your code. The more general exception is raised when something else goes wrong with your indentation, like using an unexpected number of spaces (or tabs).

[patent mica]

yes ik when that error shows up

[wide shuttle]

If you're making mistakes with your indentation that both includes the more general type and mixing spaces/tabs, it probably depends on what Python "sees" first.

[patent mica]

also i tested that dynamically with @undone hare

[patent mica]

yeah technically yes, i just forgot that coding concept, thanks for it

[sharp latch]

Is there a supported way (in 3.8+) to have a shared object that can be loaded either in a process without a python interpreter (and thus load libpython) or loaded from a python interpreter (and thus get symbols from the interpreter process). From what I've seen on OSX, loading libpython inside a python process results in weird bugs (like decref-after-free).

[pliant tusk]

@sharp latch how are you loading libpython?

(within the python process)

[sharp latch]

It's a normal shared library dependency of the shared object, so it gets loaded by the dynamic linker

[sharp latch]

https://github.com/PDAL/python/issues/75 for more context

[mossy heron]

Why did Python core developers decide to "allow" instantiation of abstract classes (classes that are inheriting from abc.ABC or abc.ABCMeta)? Is it one of those, "we are all responsible adults here. put underscore at the beginning of private properties. don't access them directly even if you can"

[spark magnet]

@mossy heron why wouldn't you be able to instantiate them? Once you define all the abstract methods, they are instantiable.

[mossy heron]

Aa, right! I think I was wrong to define abstract classes as "classes inheriting..."

Is it accurate for me to say that a class defined like this "class SomeClass(metaclass=abc.ABCMeta):" is an abstract class?

[plush patrol]

Can i get some help in #help-cherries nobody can seem to find a fix for my problem

[spark magnet]

@mossy heron yes

.. if one of your methods is abstract

[mossy heron]

Aaaah! So in Python, a class is only abstract IF one of the methods (or properties) is abstract.

Similar to abstract methods, a subclass inheriting an abstract property (declared using either the decorator syntax or the longer form) cannot be instantiated unless it overrides that abstract property with a concrete property.
https://www.python.org/dev/peps/pep-3119/

and SomeClass even if its metaclass is set to abc.ABCMeta, if it does not have abstract methods or properties are by definition not "abstract classes"....and abc.ABCMeta is probably used incorrectly...?

[spark magnet]

@mossy heron it would be wrong to use ABCMeta and have no abstract attributes, right.

[mossy heron]

Alright. Alright. Thanks @spark magnet !

[swift imp]

I wish they would change abstractmethod to break on definition and not instantiation

[spark magnet]

what would it do at definition time?

[swift imp]

Like when the class is being defined

or is that compile time?

Currently abstractmethod wont yell until you try to instantiate the class

[spark magnet]

on what condition would it raise an error?

[swift imp]

NotImplementedError

[spark magnet]

that's the name of an exception. What is it checking for?

[swift imp]

For example __init_subclass__ is called after definition but before instantiation and I've used it to check for methods decorated with abstractmethod and raise NotImplementedError that the abstract method is not implemented.

abc does that but it doesn't raise the NotImplementedError until you try to instantiate the class.

[peak spoke]

The methods can be added after definition

[spark magnet]

@swift imp but you are allowed to define a class that has abstract methods.

you just aren't allowed to instantiate it.

[swift imp]

I think it would be better if it was like this

Like maybe when you use the metaclass of abc.ABCMeta it adds __init_subclass__ to your class in __new__

[spark magnet]

@swift imp you aree talking about how to implement the check, but you haven;t said what the check should be.

there's nothing wrong with defining a class that has abstract methods.

[swift imp]

Do any attributes being decorated with abstractmethod exist is the check

[spark magnet]

that's not a bad thing: it shouldn't be an error.

[peak spoke]

That limits what you can do quite a lot

[swift imp]

yeah I see what you are saying

Because then if I ever made a class with meta of abc.ABCMeta then all classes that subclass it would have to be concrete

and I could never extend it further

[raven ridge]

exactly that, yeah.

[swift imp]

In the application I was doing, I was really enjoying when NotImplementedError was being raised immediately but I see how this can be really restricting now

[raven ridge]

you could build what you're asking for using a mixin class or a decorator.

[tidal nebula]

Hey guys I’m kinda new to this chat. Recently I’ve been trying to make a snake game in python, but came to a road block. When my snake eats the apple a curtain number of times a segment of the snake appears somewhere on the window randomly. You guys would be a god sent if you could check out my code or potentially fix my error. Keep in mind my coding skills aren’t the best, so don’t roast me too hard lol😂. Here’s the link to my GitHub: https://github.com/GavinGonzalez/Snake-Game/blob/master/main.py

[mint forge]

@tidal nebula This is not the channel for that, this is for discussion related to the language itself.
See #❓｜how-to-get-help

[lofty rover]

Imagine a list of lists with a,b,c being lists [a,b,c]

Is there a difference between going a[0], b[0], .. a[n], b[n], c[n] as compared to going a[0] .. a[n] and then b[0].. b[n]?

in terms of 'runtime'

[feral cedar]

should be about the same speed

[raven ridge]

Even though Python lists are contiguous dynamic arrays, the elements of those arrays are pointers to arbitrary heap memory. So there may be some minor cache utilization benefits from doing all the a's before any of the b's, but they'll be drowned out by the cost of all the random accesses.

[undone hare]

Actually

On modern CPUs the former will have a major increase over the latter, because of the CPU cache

The latter will have a lot of cache misses compared to the former

[flat gazelle]

python objects are not contiguous in memory anyway

[undone hare]

Aren't lists contiguous?

[flat gazelle]

lists are contiguous, but they contain pointers

those pointers do not point in a nice sequential manner

[undone hare]

Right

But assuming that the cache is big enough to still keep the array in the cache, you'll still have that big speed up

[fresh cargo]

Seems to confirm it:

>>> timeit("for i in range(100):\n    x[i]\n    y[i]\n    z[i]", setup="x=list(range(100))\ny=list(range(100))\nz=list(range(100))")
7.761224100000163
>>> timeit("for i in range(100): x[i]\nfor i in range(100): y[i]\nfor i in range(100): z[i]", setup="x=list(range(100))\ny=list(range(100))\nz=list(range(100))")
9.731940099999974

Actually this one is probably what they're asking for:

>>> timeit("for i in range(100):\n    x[0][i]\n    x[1][i]\n    x[2][i]", setup="x=[list(range(100)) for _ in range(3)]")
12.855765399999655
>>> timeit("for i in range(100): x[0][i]\nfor i in range(100): x[1][i]\nfor i in range(100): x[2][i]", setup="x=[list(range(100)) for _ in range(3)]")
15.042935899999975

Although how is the first one faster? I thought the second one would be more continuous

[raven ridge]

Wow, my initial assumption was wrong, and it makes a much bigger difference than I was expecting.

At least, in this specific case of sequential integers from a range.

[undone hare]

CPU caches are insane, aren't they

[raven ridge]

In [47]: timeit.timeit("for i in range(3):\n for j in range(100):\n  x[i][j]", setup="x=[list(range(500, 600)) for _ in range(3)]")
Out[47]: 41.70545515202684

In [48]: timeit.timeit("for j in range(100):\n for i in range(3):\n  x[i][j]", setup="x=[list(range(500, 600)) for _ in range(3)]")
Out[48]: 102.84428086597472

I expected the difference to go away using integers that aren't in the small integer cache, but actually it was magnified.

now that said - this is still maybe not a realistic scenario, since it's likely that each of the integer objects created by iterating over the range are roughly contiguous in memory, which may not be the case for an actual list of lists that isn't allocated all in one big shot like this

[tiny whale]

lmk if this is wrong channel, but I'm wondering why something like async / await was added explicitly (and asyncio to standard library) instead of having monkey patching of blocking functions like gevent does

from what I understand so far, async / await introduces the problem of functions having color. Whereas the gevent approach seems to take advantage of Python being a dynamic language

[boreal umbra]

@tiny whale the #async-and-concurrency channel would also be appropriate but discussion about the design of the language is very on topic for this channel.

I don't know what you mean by "functions having color"

[tiny whale]

@boreal umbra what I mean is that I can only await async-colored functions. If I have a function performing sync IO, I will never be able to use it without blocking in another async context

whereas based on what I've read about gevent & greenlets, it will monkey patch that sync IO function into an async version

[boreal umbra]

What is monkey patching?

[safe hedge]

Monkey patching is a technique to add, modify, or suppress the default behavior of a piece of code at runtime without changing its original source code.

[tiny whale]

let's say a library has a module a and function b in it. a.b might perform synchronous IO. In my own module, If I did a.b(), that would block whatever function I was in. So before I do a.b(), I could do a.b = ..., where ... is an async-ified version of a.b. this is called monkey patching a.b @boreal umbra

so instead of having to use a library with async support (and do await a.b()), gevent will come in and async-ify the pure python underlying modules through monkey patching to make my sync code magically asynchronous

one example is subprocess

gevent will async-ify subprocess such that any synchronous code I have depending on subprocess will now become asynchronous, without having to explicitly use async / await

[boreal umbra]

I don't have the prerequisite knowledge to make any insightful comments.

[flat gazelle]

@tiny whale explicit async is significantly easier to reason about, since you can limit the points at which you can receive an exception, get cancelled or have some other coroutine mutate data causing a data race.

[upper cradle]

Hello, guys. I need some help with my CS class assignment

[spark magnet]

@upper cradle best to get a help channel: #❓｜how-to-get-help

[patent mica]

globals are sometimes annoying i agree

but i sometimes think that there is a way to not use them, like i feel that im doing something wrong

is that True actually ?

[flat gazelle]

@patent mica that seems like it could be inside a class as attributes

[patent mica]

ah

[deft pagoda]

sometimes you can keep variables alive in a generator if you don't need a class to deal with them

In [41]: old_value = 0
    ...: def f(n):
    ...:     global old_value
    ...:     tmp = old_value
    ...:     old_value = n
    ...:     return tmp
    ...: 
In [42]: def g():
    ...:     old_value = 0
    ...:     new_value = 1
    ...:     while True:
    ...:         new = yield old_value
    ...:         old_value, new_value = new_value, new

In [43]: it = g(); next(it)

In [44]: it.send(5)
Out[44]: 0

In [45]: it.send(10)
Out[45]: 5

In [46]: f(5)
Out[46]: 0

like this example

looks like i messed it up somewhere

[tidal marten]

Do you actually need to "import" variables from the global scope?

I never declare global in my functions

[peak spoke]

The global is needed when you need to change what the global name refers to, which you mostly shouldn't be doing

[tidal marten]

I was only reading or using objects though, so maybe writing would cause scoping problem

Ohh, so that's why I never have problem with that...

Good to know

[unkempt rock]

how do u send a message like that

[deft pagoda]

```py
your code
```

[grave jolt]

Yo, Python bugs are transitioning to github? https://github.com/psf/gh-migration

[feral cedar]

👀

[peak spoke]

the pep has been there for a while, not sure how much is being done on it

[grave jolt]

Well, I only noticed it because of this notification

[undone hare]

Finally?

I have honestly been waiting for that to do some PSF contributions haha

I don't like bpo

[grave jolt]

fix typo in docs

[undone hare]

I did actually fix a broken link in the docs once haha

[wide shuttle]

They've been working on it for a while now. There was a call for volunteers for the transition working group on one of the mailing lists.

[distant quartz]

hye all - got a quick question. If I create a thread using run_executor in async, would that still block the main event loop because at some point to return the result of that thread, the GIL is going to be taken away from the main even loop/thread?

[peak spoke]

The main thread will be blocked while the other one holds the GIL, but the thread will be forced to release it at regular intervals if it doesn't yield it itself so you get the concurrency

[distant quartz]

@peak spoke So what happens if my thread is not done executing my code. The thread gets released and the computation dies?

I guess with async, the main kicker here, is that if I do run some blocking code properly, at some point the GIL is going to be taken by that blocking code executing in another thread.

But that's okay?

It's inevitable that the event loop is going to be blocked for milliseconds?

[peak spoke]

If both are executing python code, yes. The threads can switch at arbitrary instructions so unless you're doing some costly op like 1234567890**1234567890 then the threads won't block each other for longer intervals. The main difference between async and threading is that with async you control when context switches can occur and they're handled by the event loop in a single thread while threading is handled at the os and can switch anytime, but in python both temporarily block while other work is being done

[signal tide]

when would you not want a context switch?

[peak spoke]

Race conditions are far easier to prevent if you control it

For example if you had this func

def pop_if(list_, cond):
    if cond:
        list_.pop()

If you run it while using async for concurrency it'll be done synchronously and you don't have to worry about anything, but with threading the thread could switch after the condition causing multiple threads to pop from the list when they shouldn't have (when cond is dependent on the list contents)

[distant quartz]

About thread switching, since now my understanding of threads is being undermined. I was told before here that if the thread is all local, as in, no references outside, when threads switch the proper thread will be picked up and continue the correct operation but you seem to say otherwise

In addition, the code you just posted. Yeah that would run in sync since we are just using the main thread but it kind of goes back to my original question

How long is it okay to block the event loop?

50ms?

[peak spoke]

Depends on if you need to keep serving something for example, python won't really care if you block if forever but whatever you're using the async for may. I believe the default switch interval for threads is 5ms

[signal tide]

if it's a computation, you might want it to finish to completion - compared to a request/etc, where you'd want it to timeout

isn't there a run_until_complete()?

[distant quartz]

Ok that's good to know because I keep hearing don't block the event loopbfor Async but technically even a simple variable assignment blocks the event loop

Not sure if I'm making any sense but essentially I keep hearing people say don't block the event loop but everything you do blocks it for some time!!

[peak spoke]

That's right, you usually just shouldn't be blocking it for longer periods because nothing else can run while that's being done, and if you for example needed to keep some connection alive that'd be impossible while it's blocked

[distant quartz]

What do you mean by keeping a connection alive? As in using Async on a webserver?

So like a web socket connection?

That's news to me because I'm trying to implement a web socket

Wow I didn't know that holy cow, ok I'll make sure to ping the server here and there

[torpid narwhal]

I am running a long running program that over time sees RAM/CPU get to 100% then craps out. Looking at metrics before this, the event leading up to this is a 10x increase in context switches for the CPU. What does that mean?

(Python program on Ubuntu 20.04)

[weary pilot]

Low level memory leaks

Shit like that in python is rare, mostly happens in languages like C

Do u hav something like opening files inside a loop?

[torpid narwhal]

We are using https://ray.io/ and creating long-running processes to process video with opencv.

I had some opencv video captures that were not properly released. Had redis clients that were not closed as well. That is about to go out in an upcoming release.

I am curious if there is anything else on my end I could be doing or monitoring. I have a suspicion that there may be some memory leaks in the base package I am using, but I cannot fix those (since the core of that is in c++).

I am thinking about manually calling the Python garbage collector every N iterations of the main processing loop that spurns actors.

I get the memory leak part, but I do not see how that affects context switches by the cpu(s). Do you have any info on why that is? (I will admit, I do not have a great understanding of how the cpu works on that low of level.)

[grave jolt]

@minor blade Not the right channel. You can share your project in #python-discussion or in an off-topic channel. You can share the code on https://paste.pydis.com/ or on GitHub.

[minor blade]

sorry

theres so many channels lol

[grave jolt]

@minor blade Also not the best idea to share your IP address in public...

[minor blade]

yeah i dont really care

[grave jolt]

But I don't see an immediate solution

[minor blade]

ddos doesnt last forever

[boreal umbra]

As far as I know, the stdlib doesn't have a context manager for timing things

>>> with timer():
...     print('hello world')
hello world
.032432ms

Such a thing is trivial to implement, but this is the kind of thing people expect Python to have, and they love Python for it.

[feral cedar]

that's a slow ass hello world

[boreal umbra]

don't trash talk my random button mashing.

[feral cedar]

but yeah, a built in timer like that would be nice

[boreal umbra]

You were actually there when I first wanted this

with the garbage collector

[feral cedar]

oh with the linked list

[boreal umbra]

however the core devs only seem interested in adding things that aren't trivial to implement at this point. Because bloat. And I'm honestly not even mad.

[boreal umbra]

@sand goblet I like the ternary syntax

apparently some people find it counterintuitive

a = x if y else z tells me that usually a is going to be x, but there's a special case y where it's z.

did I get that backwards?

usually I have z has the special case

so y might be a not condition of some kind so that x is the expected case.

[grave jolt]

@sand goblet The conditional operator has its uses, I was just annoyed that someone was suggesting it, together with (a, b)[c], in various inappropriate circumstances, e.g. when helping a person with learning if statements.

and yes, I don't like the then-if-else ordering much, but that's more about personal taste

[boreal umbra]

I wonder if it's a language thing?

ie native spoken language

it's a bit unfortunate how Python tries to pander to Anglophones.

[grave jolt]

maybe it's a language thing, but a different one -- if you haven't used the condition operator much in other languages, it's probably natural to use the Python variant

I've used it in JavaScript/TypeScript, and I like it there

[boreal umbra]

I don't have any meaningful experience with other languages.

[grave jolt]

Speaking of syntax choices, was and and or a good idea over && and ||?

[boreal umbra]

I mean, I like it

but I'm a native English speaker

From that perspective it's a no-brainer.

[sand goblet]

Alright then, thanks for clarifying it

[boreal umbra]

The people who like && and || are the same people who think "Python doesn't look like code".

[grave jolt]

Maybe it's a general illusion that using English words helps learn and understand the syntax. Is COBOL or SQL easy to understand?

The issues come when the word makes constructs that don't make sense in Python make sense in English, e.g. I have seen this:

if foo == "a" or "b" or "c":

hundreds of times here, and I suspect that it's the familiarity of or that leads to this mistake

[boreal umbra]

no, cobol and sql are awful. But Python actually uses English keywords surprisingly sparingly.

There are fewer keywords in Python than Java.

if you don't know anything about programming and you don't speak English, "and" and "or" are probably just as arbitrary && and ||

but if you do speak English, "and" and "or" are less arcane and make programming easier to jump into.

[grave jolt]

And if you learn what && and || really stand for, you know what && and || means -- no mistery.
But if you do speak English and are trying to understand or and and, you get into this trap:

[boreal umbra]

the or gotya?

[grave jolt]

yeah

[boreal umbra]

well

!or-gotya

[fallen slate]

When checking if something is equal to one thing or another, you might think that this is possible:

if favorite_fruit == 'grapefruit' or 'lemon':
    print("That's a weird favorite fruit to have.")

While this makes sense in English, it may not behave the way you would expect. In Python, you should have complete instructions on both sides of the logical operator.

So, if you want to check if something is equal to one thing or another, there are two common ways:

# Like this...
if favorite_fruit == 'grapefruit' or favorite_fruit == 'lemon':
    print("That's a weird favorite fruit to have.")

# ...or like this.
if favorite_fruit in ('grapefruit', 'lemon'):
    print("That's a weird favorite fruit to have.")

[boreal umbra]

now they know.

[grave jolt]

but if && or || are used, there's no temptation to even fall for this

[boreal umbra]

are we sure about that?

[spark magnet]

@boreal umbra the fact that this discord has that shortcut shows that this is a persistent problem.

[grave jolt]

And if you're actually debugging code to find where the issue is, you don't have this shortcut appearing, while the code makes sense in plain English.

[boreal umbra]

I wasn't being entirely serious. But I don't have the stats to say one way or another that people can't fall into the same trap if they think || is "or" from an English semantics perspective.

[grave jolt]

yes, it would be interesting to see the statistics

[spark magnet]

i have never seen someone write if (fruit == "apple" || "lemon") { in C or Javascript

[grave jolt]

I don't lurk in non-Python spaces much, so I can't tell

[pliant tusk]

i have in back in my highschool cs class @spark magnet there were many pieces of interesting code from beginners who only partialy understood the language they were using

[feral cedar]

in c# for example, it's if cond ? ifTrue : ifFalse

maybe i have the ? and : switched, but it's all symbols, no keywords

~~except if, but like,~~it's actually the same as in java >_<

[grave jolt]

in C, C++, Java, JS it's condition ? ifTrue : ifFalse

[swift imp]

If I have a class that is list like, should it be copying items that are being placed into it, with __setitem__ ?

[grave jolt]

That's a question for #software-architecture 🙂

[swift imp]

Damn

[raven ridge]

Probably not: list doesn't, and a list-like that's different from list in such a fundamental way seems really confusing.

[swift imp]

Yeah that's why I'm gonna stop calling it a list like class

[boreal umbra]

I dunno, I think questions about the data model are very on-topic here.

[grave jolt]

This is a channel for meta-discussions about Python itself. Discussing the behaviour of a class would be more on-topic in #software-architecture

[boreal umbra]

I suppose it's also on-topic there, but the data model seems to be a frequent topic here.

Let me rephrase, it is on-topic in software-design for sure. But I think discussion about effective use Python's data model falls under Python meta-discussion.

[grave jolt]

well, okay, the question would be fine here as well.

[grave jolt]

This channel is for meta-discussions about the Python language. You should claim a help channel, see #❓｜how-to-get-help .

[pure vine]

oh yeah wrong channel sorry

[severe lichen]

I'm not native english speaker, but python keywords usually don't bother me.

Oh, with the exception of lambda.

I really like the arrow notation, x => x + 2

[patent mica]

guys help in #help-broccoli

channel seems to have lost control

@undone hare how did it work for you

like we tried !closing it

[undone hare]

I am part of the helper team

I can close any channel

[patent mica]

oh so i cant close any opened help channel

unless its me who opened it

?

[undone hare]

You can only close your own channel, yeah

The staff can close any channel

[patent mica]

oh didnt knew that

aight

[magic python]

from https://pandas.pydata.org/pandas-docs/stable/user_guide/advanced.html, is this not inconsistent with slice notation? I thought it was typically "up to but not including"

i'm wondering if there's any sort of convention / thinking that I've missed here

[undone hare]

Well, the slice notation as it is being done in CPython never uses strings, so that's a bit different here

[magic python]

@undone hare hrm ok... so this is like a pandas own sort of thing then, i guess

don't think i like that they've broken what's used across everything else... maybe it makes sense tho, i don't usually use it, guess it's more excel-ish or something

[undone hare]

You can see that as a DSL, if you're familiar with this concept

[magic python]

no, i'm not sure what a DSL is actually

[upper trout]

its giving me "invalid syntax" when i try to import discord

i tried to install with pip and pip3

oops wrong channel 😐

[patent mica]

lmao

#shrunken-discussion

[desert peak]

Domain-Specific Language

[deft shore]

Does anybody know any other websites like MIT OCW where you can get courses for free? Can you please @ me if you respond so I'll know to check

[spark magnet]

@deft shore what about edx.org ?

[deft shore]

i will check that out thank you

[solemn yew]

Is it possible for dynamic import. For example: from texts import f1, f2, f3 then it would read in f1, f2, f3, ... , fn during runtime? Instead of importing all n files?

[raven ridge]

Python doesn't have a separate "runtime", as opposed to "import time"

if you want the importing to happen later, put the import statement later.

you can import in a function, even.

[solemn yew]

whats a good way of separating imports instead of files?

i don't want f1.py f2.py ... fn.py

[raven ridge]

why not?

[solemn yew]

I have like 250 files

[raven ridge]

ok - why so many files?

[solemn yew]

well they are csv files separated by stock symbols. i don't want to import all 250 if I just want to use a small subset of them

[raven ridge]

wait, are they csv files, or Python modules? One file can't be both...

[solemn yew]

so basically I have a helper function.
TSLA = _readcsv("tsla.csv") GOOG = _readcsv("goog.csv") APPL = _readcsv("appl.csv)

i would ideally like from data.symbols import GOOG but when I try this. it imports all 250 even though I only need two

[raven ridge]

if you could live with from data import symbols and then symbols["GOOG"] or symbols.GOOG you could get what you want by making symbols a global object with a __getitem__ or __getattr__ that loads the data from the CSV if it hasn't already been loaded.

[solemn yew]

oh interesting

I'm going to try that approach. thanks

[raven ridge]

it's even possible to make what you want work - so that from data.symbols import GOOG works to dynamically load the GOOG csv into a new Python object that it then returns - but that's crazy hacky, way hackier than the alternative I'm pitching.

people will understand the approach I'm proposing 🙂

[solemn yew]

you calling me dumb?

but appreciate the help man

[raven ridge]

https://github.com/wimglenn/advent-of-code-data/blob/master/aocd/__init__.py#L51-L76 - here's a module that does it the crazy hacky way. 🙂

[solemn yew]

doesn't look all that bad

[raven ridge]

including the last line?

76, I mean?

[solemn yew]

yea i've never seen it used like that

but my code has always been pretty basic. Heck I don't even touch anything with double underscore

[raven ridge]

the last line is replacing the module with an instance of a class that shares the globals of the module, and also supports its own special attribute lookups. Trust me that it's a real hack.

[tidal marten]

Is there a way to get the path of a Python script?

Not path of the __main__ but imported module

[grave jolt]

You can enable tracemalloc and find in which module the object was created, but tracemalloc is a debugging tool, you shouldn't really enable it in production.

[tidal marten]

I found __file__, I think that should work for my use...

[grave jolt]

oh, sorry, you meant "script", I read it as "object" for some reason

[tidal marten]

Can't I do obj.__module__.__file__ for objects? Nope

Tensorflow still doesn't support 3.9, smh

[tidal marten]

I'm assuming I can't get the class of a function returned by the class' method, is this correct?

class A:
    def a(self):
         def _a(self):
            pass
        return _a

a = A()
b = a.a()
assert(b.something_something is a)

[pliant tusk]

you could if you store it there

[boreal umbra]

!e @tidal marten

class A:
    def a(self):
         def _a(self):
            pass
         return self.__class__

a = A()
b = a.a()
print(b.something_something is a)

[fallen slate]

@boreal umbra :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 9, in <module>
003 | AttributeError: type object 'A' has no attribute 'something_something'

[boreal umbra]

I'm not really sure I follow what you're trying to determine

I don't think that methods contain references to the class that they belong to (though the class has references to the methods that it has)

especially since you can do hacky stuff to give the same method (same object) to multiple classes, to say nothing of MROs.