#internals-and-peps

1 messages Β· Page 52 of 1

strange fog
#

locks, semaphores, the works

hazy anchor
#

I'm not sure I see a huge benefit

red solar
#

to using atomics?

hazy anchor
#

But I personally haven't using much shared memory in python

#

I'm familiar with them and the use in other languages, but it's never a tool I've wanted and not had in python. Guess I write different programs in it

strange fog
red solar
#

the one benefit of atomics is that they don't (usually) block

#

which is ideal within an event loop

strange fog
#

this problem keeps evolving

red solar
#

there isn't really a defined problem, i want atomics in general 🀷

strange fog
#

so it's atomics for IPC inside an event loop?

red solar
#

that's a use case yes

#

another is if i want a process safe variable in shared memory, if i put a lock there, how would c++ access said lock?

strange fog
#

you use one of the synchronization primitives provided by the kernel

#

which have their own way of being ID'd system wide

red solar
#

from python? how?

#

(i got this added in python 3.10 sorta but before that?)

strange fog
#

well for windows you can use pywin32 to access the windows api

red solar
#

through ctypes?

strange fog
#

and for linux/osx im sure there's a SYSV or POSIX api somewhere

#

pywin32 is a module that exposes the windows api to python

red solar
#

ah huh

strange fog
#

i linked one of the doc pages above

#

for creating a mutex

red solar
#

hmm... ok i guess the uses cases are for shared memory if you don't want the overhead of locks, and for process safe non blocking locks (complicated but doable)

#

(non blocking as in for use with asyncio)

strange fog
#

to have interprocess atomics like youre suggesting im pretty sure you'd need some backing from the kernel

#

and idk of any kernel that provides this

hazy anchor
#

It seems like a use case almost entirely done at the cpython level

red solar
#

all lock free atomics are interprocess safe

#

the second one yes, the first one no

#

and if they're going to be used for the second one, why not provide that freedom to everyone 🀷

strange fog
#

are you sure youre not thinking of thread safety?

red solar
#

yes

#

(c basically stole their atomics from c++)

hazy anchor
#

Describe a program that you would use this hypothetical library in

#

I'm not against it, I just want to understand

red solar
#

beyond the event loop thing?

hazy anchor
#

Atomic operations are pretty limited. Simple maths and pointer swapping. What would you be doing in asyncio that was best solved by having two corouines accessing the same variable

red solar
#

so people in asyncio rn can't use thread/process locks

#

they block

#

there are OS primitives that let you have non blocking locks, and they almost always have a file descriptor (int) as a handle

#

so now you can use this handle with poll/select/epoll whatever, to implement an asyncio friendly lock πŸ™‚

#

great for mixing threads, and event loops, since in python int is already thread safe

#

but it's not interprocess safe

#

and you cant put the file descriptor in shared memory and have it be safe as is

#

so make it an atomic int πŸ™‚

hazy anchor
#

It sounds to me like a coroutines aware lock primitive is what's required there. Maybe it can be achieved with atomic support, implemented in C using the atomics that already exist

red solar
#

this is what i'm working towards (other people helping), just realised i might need atomics for this, and thought why not propose it as part of the library, so other people can use it for their really niche use cases?

#

imo the more available directly rather than as an implementation detail, the better

#

also x86_64 has 128bit atomics which is nice πŸ™‚ (fits a uuid)

hazy anchor
#

But to do what

#

You can't do fuck all operations on a uuid

red solar
#

maybe you want to make some sort of in memory interprocess database 🀷 idk people do crazy things (definitely not me)

hazy anchor
#

Well. I'm interested but not convinced. Lucky you don't have to convince me :p if it's going on GitHub would you care to drop a link

red solar
#

i'm currently rethinking the implementation πŸ˜‚ writing an atomics library based on <stdatomic.h> is easy, but if you want 128bit atomics and stuff, you need intrinsics :/

#

i can link you to a partially complete old implementation?

strange fog
#

intersting SO post. one question

The mechanism hardware exposes for atomic read-modify-write operations is based on MESI cache coherency which only cares about physical addresses. x86 lock cmpxchg / lock add / etc. makes a core hang on to a cache line in Modified state so no other core can read/write it in the middle of the atomic operation.

what happens if it's a single core running multiple processes accessing the same address

#

does that violate the atomicity?

red solar
#

no, but i'm trying to think why

#

on x86 specifically that's not an issue, since all int operations are atomic (but not in the sense that the cache will be updated for other cores)

#

but if there's no other cores to update, then it's just always atomic 🀷

hazy anchor
#

Do I interpret the above uses a memory guard in the cache? That seems like a non starter doesn't it

strange fog
#

thats what im understanding

red solar
#

why is it a non starter?

hazy anchor
#

But it seems like in C++ the operation is explicitly supported

red solar
#

yes it does use a lock in the simplest sense (hence the lock prefix on the instruction), however it's a hardware lock at the lowest level, and it's process safe (it applies to all cores on a machine)

#

(all cores that share that cache)

hazy anchor
#

I think I'm more likely misunderstanding the cache line part. Research suggests it can work

#

I wonder if multiple cpu machines are the edge case

red solar
#

to be clear, this can still block, just realistically almost never, and if it does, it's for a tiny amount of time)

#

edge case in what? cache coherency is a multi core thing

hazy anchor
#

It must be a solved problem for atomics to work at all, so disregard me. It's interesting and not something I know about

lucid coyote
#

Is there a way to use functools lru cache for large memory requirements (e.g, graph isomorphism checking and path detection) without crashing your computer?

radiant fulcrum
#

more ram

#

ℒ️

hazy anchor
#

Swap, specifically

lucid coyote
#

Could you give an example?

hazy anchor
#

Swap as in swap space, it's disk backed memory

#

Allocate more of it

lucid coyote
#

Oh, ok

#

Thank you very much!

#

Sorry about that

hazy anchor
#

See how ya go. It's generally not that fast, but a cache for objects larger than memory than only be optimised so much. The critical point is probably to keep the keys in memory and swap out the value

lucid coyote
#

That helps a lot, thanks!

austere spruce
#

How do you guys specify your version of installable packages that you create? I want to define it one place and then being able to get the version from within my code (to set FastAPI version)?

#

I ended up doing the following:

.
β”œβ”€β”€ alembic.ini
β”œβ”€β”€ docker
β”‚Β Β  β”œβ”€β”€ Dockerfile
β”‚Β Β  └── Dockerfile.worker
β”œβ”€β”€ docs
β”œβ”€β”€ Makefile
β”œβ”€β”€ MANIFEST.in
β”œβ”€β”€ migrations
β”‚Β Β  β”œβ”€β”€ env.py
β”‚Β Β  β”œβ”€β”€ helper
β”‚Β Β  β”‚Β Β  └── __init__.py
β”‚Β Β  β”œβ”€β”€ README
β”‚Β Β  β”œβ”€β”€ script.py.mako
β”‚Β Β  └── versions
β”‚Β Β      β”œβ”€β”€ 2d1946f92088_added_users_table.py
β”‚Β Β      └── added_users
β”œβ”€β”€ README.md
β”œβ”€β”€ setup.cfg
β”œβ”€β”€ setup.py
β”œβ”€β”€ src
β”‚Β Β  └── wert_api
β”‚Β Β      β”œβ”€β”€ asgi.py
β”‚Β Β      β”œβ”€β”€ config.py
β”‚Β Β      β”œβ”€β”€ __init__.py
β”‚Β Β      β”œβ”€β”€ models
β”‚Β Β      β”œβ”€β”€ routes
β”‚Β Β      └── version.py
└── tests
    β”œβ”€β”€ conftest.py
    └── test_users.py

Where version.py simply contained __version__ = '0.1.0'
And setup.py:

# -*- coding: utf-8 -*-
"""Builds boilerplate as a package

"""
import setuptools
from distutils import util

version = dict()
path = util.convert_path("src/wert_api/version.py")
with open(path) as file:
    exec(file.read(), version)

setuptools.setup(version=version["__version__"])

unkempt rock
#

why not import version from version?

minor sinew
#

how can i test if 2 types are equal?

#

for example, if i take a type as a function input, how do i equate that input type with some other type?

open trout
#

if type(obj1) is type(obj2):

lunar trail
minor sinew
#

@open trout that's not what im asking. in that, you're getting the type from some object

#

the type im getting is a function input, so there's no object to even compare

lunar trail
#

@minor sinew As I said, move to a help channel please

boreal umbra
#

I feel like there should be a flatten function in the stdlib for flattening 2d lists

#

it's such a common problem with no universal solution.

unkempt rock
#
values = [[1, 2, 3], [4, 5], [6]]
flattened = [e for row in values for e in row]

Though I agree. a flatten() function that flattens any list or maybe a flatten(axis) would be really conventient

flat gazelle
#

for lists you can do sum(values, []), but we could absolutely use a flatten function

unkempt rock
#

wow that's actually quite clever. Never thought about that

austere spruce
#

@unkempt rock that would make it uninstallable for new users since src is not a module.

minor sinew
#

finally finished testing this function i made to build a big matrix to represent connections in a big network lol the worst part was handwriting the expected test case matrices

tawdry pine
#

Looking for advice in implementing Domain Driven Design, Event sourcing using python.

pure sinew
#

Does anyone here have experience dealing with neural networks in python?

flat gazelle
pure sinew
#

Thanks

rugged lantern
#

With cpython wouldn't each element in a list be different sizes as they don't have to be the same type?

#

(Same size in memory)

charred wagon
#

Doesn't it just store pointers to objects?

#

And pointers are always the same size

void sonnet
#

i was thinking that they're pointers, but i haven't looked so its just a guess/assumption from me.

rugged lantern
#

So it's not an actual c array then?

grave jolt
#

It is an array of pointers.

void sonnet
#

an array of point... yesh. thats. πŸ˜„

rugged lantern
#

Ok cheers

grave jolt
#

But different objects on their own, of course, can occupy different amount of space.

rugged lantern
#

Yea that makes sense

charred wagon
#

They're just somewhere in the heap aren't they

rugged lantern
#

πŸ‘

grave jolt
#

Well, all objects are in the heap, yes

#

in CPython

swift imp
#

I got a question about compiling an exe and making a gui in python. I wrote a function that takes in an excel sheet and for each row in the sheet, it uses selenium to fill out an online webform. Now I want to distribute this among the company I work for, so others can use it. If u compile this into an exe, how do I package it so the chrome driver.exe that's necessary for selenium to control chrome, is inside that .exe

#

I know nothing about doing this in python so like, idk if what I want us possible

somber halo
#

you can try PyInstaller

#

wait, just read about the need of adding chrome driver.exe too

#

I think you could try to build it as a folder and wrangle the building step in PyInstaller in order to ship that .exe

swift imp
#

Ok, thank you

split chasm
#

Yeah you can do it with pyinstaller, but it has a lot of "gotyas". Might take some time to understand hidden imports and data packages for it

golden summit
#

hey is anyone here familiar with multithreading multiprocessing* and database connections ?

#

i am trying to use multiple cores to make simulatenous inserts into a db and im getting a crash thats very odd

#
def createRandomOLD(mean, standardDeviation, numberOfUniques, SensorType,sensorID):
   
    for k in range(numberOfUniques):
        randomG = ('%.2f'%(random.gauss(mean,standardDeviation)))
        print("Creating Entry with value " + randomG + SensorType)
       
        cursor = dataBaseConnection.cursor()
        cursor.execute(
            'insert into SensorData(SensorUniqueID,SensorValue,SensorInput,SensorType,IsMalware) values(?,?,?,?,?);',
            (sensorID, float(randomG), 'Virtual',SensorType,0)
        )
        dataBaseConnection.commit()
#

so im making two threads that call this function with different input parameters.
The error that i get seems to be an issue with syncronization?

spiral willow
#

What database?

golden summit
#

its MS SQL Server

#

and the device thats running this is a raspberry pi 4

spiral willow
#

What’s the error?

golden summit
#

let me get it for u!

#

is it ok if i get a screenshot?

#

im like remoted into

#

and sometimes i get that error with no changes to the code

spiral willow
#

maybe multiple unique keys trying to be written

#

malloc screams memory

golden summit
#

worth mentioning that if i run this on my desktop i get no errors

#

and i tested it with 16 processes and it worked fine

spiral willow
#

ok

#

I think it might be Python error and not an MSSQL error

golden summit
#

yeah its odd

spiral willow
#

where is MSSQL server running?

golden summit
#

this also freezes up the sql server

#

its running on a dedicated server at a university

#

so they are hosting it with what i assume to be good hardware and all that

spiral willow
#

you have to restart MSSQL?

golden summit
#

i dont

spiral willow
#

they do?

golden summit
#

what frees up the db

#

is killing my vpn connection to the university***

spiral willow
#

are you properly committing?

golden summit
#

i beleive using try: and finally: has stopped that error

#

So im not too sure if i am

spiral willow
#

you could read back

golden summit
#

im committing every time a value is generated by the funciton

spiral willow
#

with it being RaspPi, I wonder if it can't handle the load

golden summit
#

idk if thats helpful

#

hmmmm

#

yeah im wondering

#

its a 4 core model with 4gb of ram so i expected it to

spiral willow
#

and malloc error is memory allocation errors

#

but it's not freeing up memory properly

golden summit
#

yeah the hated word from my c/c++ experiences

spiral willow
#

I'm not familiar with multithreaded library either

golden summit
#

thats fine

#

if this were your program to troubleshoot what steps would u take?

#

single threads work fine* idk if i mentioned that

spiral willow
#

I can tell you that MSSQL can handle numerous applications dumping garbage in it

#

we use it at work all the time

golden summit
#

yeah the server is set to unlimited connnections

spiral willow
#

instead of multithreaded library

#

run it with just two SSH windows and python app (no multithread)

#

maybe third SSH windows to watch top to make sure you don't swap and explode

golden summit
#

yeah so thats where we run into an issue

#

we have been told that it should all run from one script

#

and on a rpi

#

but that does work

#

we have tried that with like 4 seperate terminals

spiral willow
#

so make a script that runs 4 python3 app.py?

golden summit
#

so 1 script that launches all the others?

#

i think we tried that and it finishes the scripts in order and not simulataneosly

spiral willow
#

I'm not familiar with Python launching stuff

#

I use Powershell for my OS level work

golden summit
#

ah ok

#

thats fine i appreciate the help

#

i think we may have to do that and move on

spiral willow
#

I wonder if Multiprocessing thread is keeping track or something and in middle of keeping track, runs out of memory and crashes

#

are you spawning or forking?

golden summit
#

hmm im not sure what spawning and forking is

#

i believe im spawning if youre referencing the creation of the thread

spiral willow
#

ok

#

yea, I mean multithreading this may or may not speed things up

#

probably will because SQL IO is long

golden summit
#

our intention is to be able to change the rate of inserts per thread,

for example thread 1 inserts every 5 seconds while thread 2 inserts every second

spiral willow
#

you could see if pyobdc supports async

#

:butwhy:

golden summit
#

we are emulating sensors with varying poll rates

#

but im starting to think more and more just use seperate scripts and say its easier this way

spiral willow
#

you should make time between inserts an argument for the script

golden summit
#

thanks for the help!

#

ill look into it

spiral willow
golden summit
#

i see

#

thats useful

gleaming rover
#

in general, I think argparse is preferred to getopt

mystic cargo
#

If I'm not wrong, getopt is depreciated in favour of argparse

boreal umbra
#

Isn't None fundamentally just a sentinel with a falsey bool value?

unkempt rock
#

can anyone takea look

peak spoke
boreal mantle
#

kind of just looking for a simple validation library that i can write my own custom rules for and isn't too complicated but would be a lot of libs to test out by hand lol

hollow crane
#

rip advanced-python

oblique crystal
#

it's only a name change

paper echo
#

@boreal mantle i use marshmallow and jsonschema for this

#

I actually really would love a library that automatically generates a marshmallow schema from a jsonschema

#

Or vice versa although that will entail some restrictions on the marshmallow schema

hybrid shell
#

Wait wait..

#

#advanced

#

ah..

narrow kettle
#

does python have a built in code generator? i need to generator some repeated code

flat gazelle
#

no, there are no convenient macros. Your best bet is making it in a loop or possibly eval

narrow kettle
#

hmm, when you say eval, can i generate code that can then be used in the project it self

pliant tusk
#

@narrow kettle would it not work simply as a function?

narrow kettle
#

well its generating functions

#

if thats what your asking

#

in C# id use a source generator

#

to get an idea what i want

pliant tusk
#

can i see what your use case is?

flat gazelle
#

what is the actual goal here. It should be possible by setting module attributes or similar depending on what you are doing

narrow kettle
#

i had an idea to generate decorators for all the events in my messaging system

#

ive been working on this for some time, with several different ideas but none have panned out

flat gazelle
#

so you have a bunch of event classes and want decorators for them?

narrow kettle
#

i have a bunch of events

#

the events are just strings

#

i need to generate decorators for each one

pliant tusk
#

you can make a function inside a function

narrow kettle
#

yee thats how decorators work

pliant tusk
#

like ```py
def x(num):
for i in range(num):
def f():return i
yield f

narrow kettle
#

and how does this help me?

#

i need to generator decorators lol

pliant tusk
#

so you could gen all your decorators like that?

narrow kettle
#

can i then use them in the project

flat gazelle
#
for event_string in events:
    def decorator(): ...
    globals()[f'on_{event_string}'] = decorator
``` do be careful that you cannot use event_string inside the decorator function
#

would be the shitty metaprogrammy way

pliant tusk
#

or you could create a dict then add that dict into your modules scope, or put them as properties on a class

narrow kettle
#

i want the freedom to have the event name and the function name differ

flat gazelle
#

ye, this would be better as class attributes.

narrow kettle
#

uhh

#

elaborate on that?

#

wdym class attributes

charred wagon
#

You can assign new attributes to objects after they've been defined and even instantiated.

narrow kettle
#

ya thats what ive been working on

paper echo
#

Python can generate source from AST no?

pliant tusk
#
class events:
  ...

setattr(events, "on_event_2", func)```
narrow kettle
#

i need to apply the decorator at instance creation

paper echo
#

Cant do that J

#

It's a common request

narrow kettle
#

you can

paper echo
#

Not possible with the way classes are defined

narrow kettle
#

i did it yesterday, the problem was it doesnt work with decorator args

paper echo
#

You have to do it in init

#

Not with a decorator

narrow kettle
#

one sec ill show you

pliant tusk
#

i mean you can hook class creation

charred wagon
#

Can't you do it with __new__ or something?

pliant tusk
#

technically

final whale
#

hmm

flat gazelle
#

oh, do you have the

class A:
    @self.dec
    def fun(self):
        ...
```problem
paper echo
#

Yes exactly

narrow kettle
#

no you have to override __get__

#

in the decorator class

paper echo
#

The point is it's not built into the language and you have to use your own introspection logic to get it to work

flat gazelle
#

ye, there are a few ways to get around that

paper echo
#

There are several ways to implement it

#

But none of them are built into the language

#

Personally I think overriding attribute access is the wrong way to go about it

#

You should do what the attrs lib does imo

#

"Mark" a method with a decorator

final whale
#

that sounds like the thing that dpy uses for cog listeners, maybe you can take a look at it

paper echo
#

Then in __new__ replace the marked methods with the decorated version

#

@final whale that too

narrow kettle
#

one sec let me sort back through my git history to find what im talking about lol

#
from functools import partial
class Decorator(object):
    def __init__(self, func):
      self.func = func

    def __call__(self, *args, **kwargs):
      print(f'__call__ {args}')
      return self.func(*args, **kwargs)   

    def __get__(self, instance, owner):
      print(f'__get__ {self}')
      return partial(self.__call__, instance)

class ham:

  @Decorator
  def eggs(self):
    print('hello from eggs')
#

this allows you to get the instance ref

#

which you can then use to assign the decorated method reference

#

originally i wanted to have it so i could do @Set('the_event_name')

#

but for the life of me i cant figure out how to get the acutal method reference and not just the object instance

#

but if i can generate those for every event then i can just have a non parameterized decorator for each event

#

@messenger.some_event

#

instead of @Messenger.Set('some_event')

#

whoop

pliant tusk
#

@narrow kettle just make Set return Decortator

narrow kettle
#

i tried all manner of versions of that

#

could not get it to work like i wanted

paper echo
#

The class has to know about the decorator at object instantiation time

pliant tusk
#
def set(name):
  class Dec:
    ...
  return Dec
narrow kettle
#

like that code up there works for what i need, i just would need to have a decorator per event

#

how would that work chil?

flat gazelle
#

could do that with __getattr__

pliant tusk
#

@narrow kettle you can return a non-initialized class from the function so customize the class by event name

#

and just return the class

narrow kettle
#

basically if i make it a general decorator for all events then i need a way to get the acutal method reference from the decorator

pliant tusk
#

and do whatever instance hacks inside that class

narrow kettle
#

i want zero name conventions

#

thats the kicker

pliant tusk
#

wait so what result do you want?

narrow kettle
#

ya it would be easy if i could just make it so the name of the event has to match the method name

paper echo
#

@narrow kettle look at how its done in attrs and discordpy

narrow kettle
#

what file is that in

final whale
#

discord/ext/commands/cog.py

narrow kettle
#

wait so what result do you want?
i want the flexibilty to do something like this ```py
class SomeService:
@messenger.Set('some_event_name')
def some_random_name(self):
pass

#

i can do this rn

#
class SomeService:
  @messenger.some_event_name
  def some_random_name(self):
    pass
#

but if i did that id want to generator all the event decorators instead of typing them

#

hence the generator question

#

ahh ok, this d.py thing might be what i want

#

hmm

final whale
#

maybe with composition

narrow kettle
#

but out of curisoity IS there a good way to generate python code?

#

that can then be referenced later

pliant tusk
#

i mean you can generate AST and compile it/convert it to source code

#

and you can use eval

#

but there isnt really any good way

narrow kettle
#

does eval add those types to the current running assembly

pliant tusk
#

exec would

narrow kettle
#

interesting

#

ill look into this return the decorator class ty

final whale
#

so far, here's what I could come with

#
import functools
import typing

Result = typing.Union[typing.Any, Exception]

class EventHandler:

    def event_handler(self, name: str = None, /):

        def wrapper(func: typing.Callable):

            handler_name = name or func.__name__

            if curr_handlers := getattr(self, handler_name, None):
                curr_handlers.append(func)

            else:
                setattr(self, handler_name, [func])

            @functools.wraps(func)
            def wrapped(*args, **kwargs):
                return func(*args, **kwargs)

            return wrapped

        return wrapper

    @staticmethod
    def _call_handlers(handlers: list, *args, **kwargs) -> typing.Generator[Result, None, None]:
        for handler in handlers:
            try:
                yield handler(*args, **kwargs)
            except Exception as error:
                yield error

    def dispatch(self, event_name: str, *args, **kwargs):

        handlers = getattr(self, event_name)  # might raise

        return tuple(self._call_handlers(handlers, *args, **kwargs))
narrow kettle
#

just saw this @final whale i appreciate it ill take a look

grave jolt
#

@paper socket This is not a help channel, please read the channel description. If you want hep with a specific question, check out #β“ο½œhow-to-get-help

#

Personally, I don't see why a chatbot is better than a wiki-like page

#

Python is often used for chatbots, so if you know Python, use it.

#

There is no single framework for bots since they work very differently on different platforms. You can search for a Python library for your specific platform.

#

@final whale I would rather do name: str = "", or name: Optional[str] = None. I'm not very fond of type annotations everywhere (it's obvious that name is a string, and func is a callable), but if you are, make them accurate.

#

Also -- what's the point of the wrapped function? You can just return func.

#

Another possible issue: name clash. handler_name could clash with some other attribute, so maybe put the handlers in a _event_handlers attribute that would be a dict.

cloud crypt
#

I am still with None-aware operators in my mind lemon_sentimental

red solar
#

oh we changed the channel name

#

lol did you guys see what went down in python-ideas mailing list?

true hollow
#

If "explicit is better than implicit", why there is no @instancemethod, for example?

cloud crypt
#

because it is default behavior I guess

red solar
#

explicit is better than implicit if implicit could be ambiguous

#

which in this case it's not

cloud crypt
#

there should be one β€” and preferable only one β€” obvious way to do it

#
macro_def ignore_error($line, $errors):
    try:
        return $line
    except $errors:
        return

x = ignore_error!(1 / 0, ZeroDivisionError)  # will be None``` my brain is throwing weird ideas now ![lemon_thinking](https://cdn.discordapp.com/emojis/680222129744380010.webp?size=128 "lemon_thinking")
true hollow
#

are you rusting?

hazy anchor
#

Oxidising :D

cloud crypt
#

rusting very

#
def _ignore_error_fn():
    try:
        return 1 / 0
    except ZeroDivisionError:
        return

x = _ignore_error_fn()``` can get translated to this I guess
true hollow
cloud crypt
#

it has no practical use in python imho

final whale
#

Thank you for the suggestions, will fix @grave jolt

cloud crypt
#

who πŸ‘€

#

ah

#

I see

red solar
#

you know how _ prefix makes variables private in the sense that you shouldn't access them and if you do and break something it's your fault...

#

does the same apply to dunder methods?

flat gazelle
#

if you mean manually calling dunders, it should be avoided, but is needed for things like __debug__, __file__, __name__, ...

red solar
#

ok but someone manually calling __new___ and __init__?

flat gazelle
#

generally a bad idea unless you are doing super().__init__. Sometimes needed when doing more complex with classes

red solar
#

ok, so no one should be calling them in a way that's semantically different than using standard syntax like super().__init__ and Class()?

flat gazelle
#

well, shouldn't is a strong word. Abusing the object system in python is pretty common for various libraries, but if you are just working with normal, not horribly mangled python objects, you should just use Class()

languid dagger
#

super().__init__ and Class() both do different things so I wouldn't try equate them

tawdry gulch
#

Class() calls new as well right?

flat gazelle
#

they are entirely different

tawdry gulch
#

Darn, that new is supposed to be dunder*

languid dagger
#

You should use builtins to call the dunder methods where available

lst = [1, 2, 3]
s = str(lst)  # good
s = lst.__str__()  # bad```
#

Class() calls __call__ of the Class type and then __new__ and then __init__ and possibly others (for example dataclasses have a __post_init__)

red solar
#

ohhh that's what __call__ does

#

was very confused, cuz in c++, you're not allowed to have static operator()(...)

languid dagger
#

You can define your own __call__ for instances

class A:
    def __call__(self, x, y):
        return x + y

>>> a = A()
>>> a(1, 2)
3```
red solar
#

but not as a classmethod?

flat gazelle
#

no. You need a metaclass for that afaik

red solar
#

ah ok so it is like c++

long pulsar
#

I think this is more of an opinion question so I don't want to post it in one of the help channels.
What I'm doing atm is: I start from a list of keys, these are also the keys in my database. I have a data object in which I store information, and that contain some helper methods like representaion and serialization.
Then I have a lookup objects. The reason I want to maintain state is that it can be called several times during the overall process and I can often resuse the results. Also yeah, just felt cleaner to me.
(below you find a minimal example)

However I like being explicit. And for me that also means I like to return things. In gerneral none of the methods would have to return, since I'm always mutating objects. But I could come up with one in each case.What do you think?

#
class MyDataClass:
    def __init__(self, key, datum01=None, datum02=None):
        self.key = key
        self.datum01 = datum01 
        self.datum02 = datum02
    
    def to_json(self):
        return json.dumps(self, default=lambda o: o.__dict__, sort_keys=True,
                          indent=4)

def get_data_from_db(keys):
    samp = [['001', 'foo', 666], ['002', '#googleMurrayBookchin', 42], ['003', 'barfooo', 2]]
    return samp

class MyLookupClass:
    def __init__(self, config):
        self.lookup_dict = {}
        self.config = config

    def prepare_lookup(self, data_objects: List[MyDataClass]):
        keys = (obj.key for obj in data_objects)
        for key in keys:
            self.lookup_dict[key] = ''

    def populate(self):
        db_response = get_data_from_db(self.lookup_dict.keys())
        for row in db_response:
            self.lookup_dict[row[0]] = row[1:]

    def update_objects(self, data_objects: List[MyDataClass]):
        for obj in data_objects:
            data = self.lookup_dict[obj.key]
            obj.datum01, obj.datum02 = data[1], data[2]


from look_up import MyLookupClass, MyDataClass
def this_happens_in_flask(keys: List[str]):
    config = {}
    look_up = MyLookupClass(config)
    data_obj = [MyDataClass(key) for key in keys]
    look_up.prepare_lookup(data_obj)
    look_up.populate()
    look_up.update_objects(data_obj)
flat gazelle
#

that seems quite overcomplicated. Why does one need to call 4 functions to get the data. I would just do

class LookUp:
    def __init__(self, keys):
        self._lookup = {}
        self.keys = keys
    def get_object(self, key):
        if key not in self.keys:
            raise KeyError
        return self.lookup[key]
    @property
    def lookup(self):
        if not self._lookup:
            self._lookup = {key: MyDataClass(key, *data) for key, *data in get_data_from_db(self.keys)}
        return self._lookup
def this_happens_in_flask(keys: List[str]):
    lookup = MyLookupClass(keys)
    results = lookup.lookup
```unless there are other requirements which mandate having so many functions for fetching data
long pulsar
#

So yes there is a bit more to it of course. Populate for example also does some postprocessing on the data. And the update process is more complicated. And it's more about the general question not the application here. Imagine the complexity is warranted. Would you explicitly return or mutate?

Also I've recently watched Brandon Rhodes on Clean Architecture. And I kinda tried to follow the priciples by not burrying my I/O opperation. And the person I talk most with about coding is a reactive kotlin programmer that follows that shit pretty zealously lol

red solar
#

@raven ridge (or anyone else who does c stuff), you think 3 pointers worth of indirection is too much for a python function call? (like atomic_compare_exchange_strong)

gloomy rain
#

@thick topaz This channel is for indepth discussion about the Python language. If you want help or feedback, check out #β“ο½œhow-to-get-help

thick topaz
#

oh ok

long pulsar
#

@gloomy rain since you are here, would you say my question if in depth enogh or shoudl I also rather ask in a help channel.

gloomy rain
#

I think at face value it seems more like a help channel situation.

#

If you need help in general, I think it's better to use a help channel.

#

If you just want to discuss something related to the Python language, this channel is more suitable.

#

But it's a bit of a gray area.

#

You're asking for a code review here basically.

unkempt rock
#

Hi. I want to count the possible multiples of 3 in range 50 to 500. How can I do that?(I'm a beginner so the simplest way possible please )

gloomy rain
#

@unkempt rock This channel is for indepth discussion about the Python language. If you want help or feedback, check out #β“ο½œhow-to-get-help

slim wing
#

how did Guido created python???

#

and is it really very easy creating a language as such ?

gloomy rain
#

He presumably wrote the first draft of an implementation in C. No, implementing a practically useful programming language is not easy.

slim wing
#

ooo

undone hare
#

For the compiling part, Guido mostly followed the parser and lexical analyser implementation outlined in the dragon book, if you are interested

rugged lake
#

how did he even have the effort to do it

narrow kettle
#

Same way anyone has the effort to write a language

rugged lake
#

how to have effort

narrow kettle
#

Same way as anything else you work hard at πŸ€·β€β™€οΈ

true hollow
#

I doubt the first python code was executed the first day of its developent

#

Cresting a programming language (the syntax and rules) is easy, just have imagination and done

#

But creating the interpreter/compiler is hard work

#

You need to scan if the syntax is okay, and then parse it to convert it to machine code or bytecode

narrow kettle
#

Ehh, tbh the hard part is the syntax and the idea lol

#

Basic Parsers and lexers aren’t THAT hard

#

And there’s a million resources on how to do them

#

The hard part is having a consistent idea for what you want

true hollow
#

and btw to convert it to machine code you need assembly knowledge

#

how are you supposed to write a C compiler from scratch without assembly knowledge

#

welp, with python is different

#

the result is bytecode

narrow kettle
#

theres many different ways to approach such things

#

some more difficult then others

#

but the actual coherent idea/spec is the hard part

#

the other stuff is tedious but can be learned

true hollow
#

python is at the same way powerful but sometimes inconsistent

narrow kettle
#

its not perfect no, no language is. especially one as old as python

#

things change, ideas evolve

true hollow
#

pep8: classes must start with uppercase and follow camel case
str, int, or bool go brr brrr

narrow kettle
#

or float

spice pecan
#

builtins go brr

narrow kettle
#

but that differentiates them as primitives persay

#

same in C#

#

int is just a alias for Int32

spice pecan
#

I think lowercase in built-in class names is fine, the only gripe I have is some inconsistencies in methods/dunders

#

In case of str, most if not all method names are just joined, while other classes tend to have underscores separating the words

#

Similarly, some dunders just join the words, like __getattribute__, while newer ones seem to use underscores, like __set_name__ and __class_getitem__

peak spoke
#

That's mostly because they came before the style guides or depend on something that doesn't follow python's naming

spice pecan
#

I understand why it's like that, and I don't mind it that much, but it's still a little annoying at times

flat gazelle
#

It would probably also be confusing to beginners which builtins are capitalized and which are not

remote lantern
#

isn't python's variable naming snake_case?

flat gazelle
#

ye, but str, int, type, bool, ... is a class, which follow PascalCase

remote lantern
#

interesting

#

functions are pascalcase too, right?

flat gazelle
#

everything but constants and classes is snake_case

remote lantern
#

ah

#

i know constants are in allcaps

flat gazelle
#

functions, modules, attributes, methods, variables, packages are all snake_case

remote lantern
#

so it would seem

#

PEP8 is a handful

true hollow
#

logging module flashbacks judoon

paper echo
#

why don't multiprocessing.Process objects support the context manager protocol

red solar
#

i thought lock supports it?

#

not sure how the process would support it tho?

paper echo
#

process has a close method

#

so you'd expect it to have enter and exit

spark magnet
#

@paper echo i don;'t have an answer, but: contextlib.closing() could help

paper echo
#

yep, thats a handy one

stable grail
#

@paper echo you can also look into this

#

!d multiprocessing.managers.SyncManager

fallen slateBOT
#
class multiprocessing.managers.SyncManager```
A subclass of [`BaseManager`](#multiprocessing.managers.BaseManager "multiprocessing.managers.BaseManager") which can be used for the synchronization of processes. Objects of this type are returned by `multiprocessing.Manager()`.

Its methods create and return [Proxy Objects](#multiprocessing-proxy-objects) for a number of commonly used data types to be synchronized across processes. This notably includes shared lists and dictionaries.

`Barrier`(*parties*[, *action*[, *timeout*]])[](#multiprocessing.managers.SyncManager.Barrier "Permalink to this definition") Create a shared [`threading.Barrier`](threading.html#threading.Barrier "threading.Barrier") object and return a proxy for it.

New in version 3.3.

`BoundedSemaphore`([*value*])[](#multiprocessing.managers.SyncManager.BoundedSemaphore "Permalink to this definition") Create a shared [`threading.BoundedSemaphore`](threading.html#threading.BoundedSemaphore "threading.BoundedSemaphore") object and return a proxy for it.... [read more](https://docs.python.org/3/library/multiprocessing.html#multiprocessing.managers.SyncManager)
paper echo
#

huh

#

this looks like it's meant more for e.g. wrapping non-multiprocessing-aware things, no?

#

e.g. sharing a value or array or threading.Lock across processes

unkempt rock
#

Does close actually do anything tho?

paper echo
#

evidently its not necessary

languid dagger
#

Is there a reason you can't join?

paper echo
#

@languid dagger they wanted to run 2 processes then kill both after one of them finished

languid dagger
#

Maybe you want concurrent.futures.ProcessPoolExecutor so you can wait for futures

radiant fulcrum
#

tbf if its not cpu or io bound async might be better also

paper echo
#

@languid dagger can you wait for "first completed" with the executor?

#

i know you can with asyncio

languid dagger
#

I believe that works, I haven't used concurrent.futures

paper echo
#

ahhh nice

#

its the same as asyncio.wait

#

very useful

unkempt rock
#

ummmm

clever schooner
#

Hey, I got a problem with a small double for loop problem. Im getting a "float" object not iterable error. However, I have no idea what is wrong. Here is what I have ``` for i in range (n):
avg += x[i]
avg*=(1/n)

m=[2,3,4]
new_m=[]
for j in m:
    for i in range(11):
        m+=(x[i]-avg)**j
    m*=(1/n)
new_m= new_m.append(m)```
#

If I could get some help that would be much appreciated. Thank you in advance

remote lantern
#

you're trying to iterate through m (for j in m)

#

but you're adding to m

#

so is it an iterable or not

#

maybe try naming your variables something less confusing?

feral marsh
#

oh yeah

#

you forgot to do the ()

clever schooner
#

brackets didnt change anything in regards to the error :/

#

@remote lantern it seems like I understood what u mean but It came up with a different error later down

remote lantern
#

hmm

clever schooner
remote lantern
#

that sounds... concerning

clever schooner
#

Ill send u the error and the problem 1 sec

remote lantern
#

it means something's returning None

#

hmm

clever schooner
#

this is what I was trying to perform

remote lantern
#

strange

clever schooner
#

do u have any idea?

remote lantern
#

i think it's the line new_m = new_m.append(m)

#

append() returns None

#

so when you try to give that value to new_m itself i think new_m turns into a None object?

#

that's my best guess

#

just do new_m.append(m) and it should work

#

also may i recommend learning about list comprehensions

#

very useful and may compact your code

clever schooner
#

just do new_m.append(m) and it should work
@remote lantern same error came up :/

#

nvm i got it

#

indentation error

grizzled vigil
#

@clever schooner @remote lantern For future reference, such conversations are better placed in the help channels, rather than #python-language. As per the description, this channel is intended for discussion of advanced language features, PEPs, etc; not for providing troubleshooting assistance.

remote lantern
#

will do

#

was only talking here because chat seemed dead and they needed help

grizzled vigil
#

That's okay. If it occurs again, you can redirect them to a help channel, then have them @ mention you if you are willing and able to assist with the problem.

remote lantern
#

alright

unkempt rock
#

Okay so i need some help regarding scheduling my tasks
so at first i have one ,

my_list = ['94230', '40737', '06959', '62168']

now i want to print the first value of my_list....which is '94230'
the i want to keep the same code for 2 days/48hrs then i want to delete
the first item of the list which is '94230' then print the second item
of list which is '40737' and keep on following the same pattern....

so how should i do that?

red solar
#

a loop with time.sleep()?

#

but also not the right channel (i feel the channel name might be misleading)

unkempt rock
#

@red solar its regarding python stuff right?

brave badger
#

Discussion about these concepts. I think your question may be suited better for one of the topical channels or a help channel.

unkempt rock
#

@brave badger Sorry didnt read the pins

quick elbow
#

I want to make a OS with Python, this will be a simple OS, like ms-DOS (Maybe more simple lol). Can I do it with Python?

true hollow
#

@quick elbow with cpython definitely not

#

also if you want to make an os like msdos you must learn x86 assembly

quick elbow
#

is it hard?

#

x86 Assembly*

true hollow
#

for me nope

quick elbow
#

I just want to make a ultra simple os

true hollow
open trout
#

TempleOS πŸ‘Ό

true hollow
#

also wdym by "simple os"

#

just a hello world?

quick elbow
#

not just hello world

#

I was making a .py for this os

true hollow
#

filesystem, keyboard input, screen output, api?

#

btw you can't create an os with cpython

quick elbow
#

hmm

#

wait lemme look to assembly

true hollow
#

@quick elbow *x86 assembly

#

there are many assemblies

#

powerpc, arm, x86, mips, 6502, jvm

#

AVR, RISC-V...

#

m68k...

#

w8 this is offtopic flush

quick elbow
#

my project was:
it has a simple mail system that can send mail to gmail/outlook . blah blah...

#

and

true hollow
#

thats not simple lol

quick elbow
#

it dont have a ui

#

thats not simple lol
@true hollow I did it with python in vscode πŸ€¦β€β™‚οΈ

true hollow
#

you need to implement:
β€’ network driver
β€’ some keyboard driver and screen driver
β€’ and then create the mail client

#

oh lol

open trout
#

"just a simple OS" famous last words

true hollow
#

no os is simple lol

#

I think you mean with simple os "a basic bootloader"

quick elbow
true hollow
#

that's not hard lol

#

Basically move data to registers and trigger some interrupt

quick elbow
#

if I can create a msdos like OS I will buy a soooo old computer lol

true hollow
#

you don't to lol

#

you just need an x86 computer with bios

quick elbow
#

is x86 is byte?

true hollow
#

what?

quick elbow
#

waitwait

#

my english bad sorry

true hollow
#

May we move to DMs and let moderators purge this offtopic conversation?

quick elbow
#

gogo dm

unkempt rock
#

Hey, can anyone tell how to keep bot connected forever?

#

Cuz whenever close the application in which python I running, bot goes offline

gloomy rain
#

@unkempt rock This channel is for indepth discussion of the Python language. If you need help, you should check out #β“ο½œhow-to-get-help.

unkempt rock
#

Oh ok

#

Thx

subtle pike
#

@quick elbow sure you can, you will need to:

  • implement python yourself in C or assembly
  • implement a PE/ELF loader in C or assembly
  • implement your own executable format then write a compiler mod to compile the exiting CPython source into it (you will need to account for expected platform resources like threading)
#

the issue is that python, itself, isn't granular enough and relies on a lot of resources behind the scenes

#

for example, python heavily relies on a garbage collector which requires a heap, which requires a memory manager, etc

#

@true hollow assembly itself isn't hard but to implement an actual OS is

#

there is a reason why each instruction listing in the intel manuals has barely a couple sentences describing it but has entire volumes dedicated to memory management or execution modes

#

@unkempt rock ?

flat gazelle
#

It would be much easier to lean a language more suited to OS dev (C, go, possibly rust, zig, odin) than to make an OS in python.

quick elbow
#

@subtle pike thanks for info

unkempt rock
#

@true hollow why are you trying to convince people x86 is easy πŸ˜‚

true hollow
#

because it is

#

modern programmers portrait is as a hell language only for people with 200 iq

flat gazelle
#

getting things done is ASM is more tedious than difficult

true hollow
#

depends what are you doing

#

if you are creating a python module, well, good luck

flat gazelle
#

because you do not have cheap abstraction that other languages give you

unkempt rock
#

Doing stuff in asm is tedious, and you need to know a fair amount and can’t make any mistakes (python is significantly easier in that regard) - doing stuff well in asm takes a whole lot of in depth knowledge of your target architecture

#

Maybe you could say some RISC ISAs are simple....

#

But I wouldn’t say that about x86

true hollow
#

welp for me x86 assembly is easy
gimme docs and what i need to do and done

unkempt rock
#

The docs are the huge book that is the intel developer manual

#

Right of the bat that’s significantly harder to read than python docs

#

That and agner’s stuff

true hollow
#

read x86-64 amd docs instead

unkempt rock
#

Yeah doesn’t look significantly simpler

golden imp
#

what are people's thought on the overrides library, or explicit overrides in general?

unkempt rock
#

Seems pretty cool

clever skiff
#

Someone indicates a free Python OOPs concepts course, please? Thanks

#

Sorry for my english

magic python
#

@clever skiff wrong channel

#

!resources

fallen slateBOT
#
Resources

The Resources page on our website contains a list of hand-selected learning resources that we regularly recommend to both beginners and experts.

narrow kettle
#

not the correct channel

grave jolt
boreal umbra
#

Why are the types returned by keys, values, and items for dict not lists?

raven ridge
#

Making them lists would result in slower code because of the extra copying

#

Plus they have a feature lists don't, they update as the underlying container does.

boreal umbra
#

I don't understand the added utility of them updating along with the dict itself, since you can call those methods at any time

#

Unless they're just a window into the data structure and don't actually cause anything new to be created

charred wagon
#

Yes, that's what they are afaik

boreal umbra
#

And the fact that they have the external behavior of self-updating is a side effect

#

In that case, why are they methods rather than at the ivy?

#

Attributes?

charred wagon
#

I don't know

#

It could be a property I guess

boreal umbra
#

Or maybe those methods have arguments that I don't know about.

charred wagon
#

But they didnΓ€t exist back then

boreal umbra
#

Did the code underlying properties exist though?

#

I think I remember reading that properties are a utilization of some language feature where you have untouchable objects that implicitly affect what the dot operator does for classes that use it.

charred wagon
#

Not sure, I suppose that is a good point. They're implemented in C so they wouldn't need to depend on the public properties API

#

But I think I misunderstood your question

#

Yeah, I don't really see why the view would need to be re-created each time if it's dynamically going to update. It could just be created once and stored as an attribute

raven ridge
boreal umbra
#

I didn't get that impression based on your answer

charred wagon
#

Ah well that explains it all

#

Backwards compat then

boreal umbra
#

Makes sense.

swift imp
#

Returning lists would be inefficient

#

That's why

#

Plain and simple

boreal umbra
#

I don't know that it's plain and simple

raven ridge
#

I think I remember reading that properties are a utilization of some language feature where you have untouchable objects that implicitly affect what the dot operator does for classes that use it.
@boreal umbra You're describing "descriptors"

boreal umbra
#

Since lists are the primary structure for ordered data

raven ridge
#

yes, but copying is slow. It is that simple.

charred wagon
#

They could cache the view objects but presumably they don't since it's not worth it.

boreal umbra
#

Right, but most python users don't know how the language is implemented, so it wouldn't be immediately obvious that using lists would be problematic in this case.

swift imp
#

What's faster in C, passing an entire array or passing a pointer to that array and an int indicating the length?

raven ridge
swift imp
#

It's really that simple

raven ridge
charred wagon
#

It would be okay if multiple calls to d.keys() (etc.) returned the same object, since the object's only state is the dict to which it refers. Is this worth having extra slots in the dict object for? Should that be a weak reference or should the d_keys (etc.) object live forever once created? Strawman: probably not worth the extra slots in every dict.

#

Neat thanks

boreal umbra
#

Well, I'm only objecting to characterizing this as plain and simple from the perspective of those who don't know about or aren't interested in low level implementation details.

raven ridge
#

yeah - I agree that it might be non-obvious, but it is "plain and simple" in that the one word answer "efficiency" is correct.

boreal umbra
#

But yes, descriptors are what I was thinking of. I've been meaning to learn about those, so thanks for reminding me of the name

#

Wow, 2006

raven ridge
unkempt rock
flat gazelle
#

Probably not, but I can't quite tell what they claim, so it is hard to tell. Python definitely does not do anything other than tokenization key by key though

unkempt rock
flat gazelle
#

Python does have an ast which it does use (though linters afaik use a modified one due to comments and other things the default one does not keep). Hell, python gets compiled to bytecode before running.

#

And that compilation is cached for future use to speed up loading

unkempt rock
#

the _pycache_?

flat gazelle
#

Ye

unkempt rock
#

and also, is that person correct?

#

or wrong

flat gazelle
#

Wrong from what I can tell. I mean, the translation from tokens (you can see how those get made with the tokenize module) to ast obviously goes over every token. In regards to parsing/compilation python is not all that different from java and some C impls

#

Python does have a grammar, it is somewhere in the docs

unkempt rock
#

aint gonna stop xd

#

I am kinda done with this person

#

xd

last pollen
#

I'm not quite sure what they mean by "scan the script by each keyword"

flat gazelle
#

ye, he clearly has no clue what he is talking about. The python tokenizer is quite normal

last pollen
unkempt rock
#

Yeah, ok

#

This guy is really trigerring me

#

xd

flat gazelle
#

well, the python lexer is a bit irregular due to handling indentation

unkempt rock
#

I mean, other than that

#

Indents dont really matter

#

he is like comprehensions suck and idk what not

grizzled vigil
#

At some point, it's just better to step away from the conversation when you realize that it's not going anywhere productive.

unkempt rock
#

Yeah

#

Thats what I did

#

xd

pseudo patio
#

can someone help me

#

Why VS code doesn't read my txt file

brave badger
boreal umbra
#

I have an idea

#

suppose this syntax were legal

#

from typing import Dict, List, * as t

#

Dict and List are brought directly into that namespace, and then the whole module is imported as t

timber osprey
#

hey guys

#

new here

boreal umbra
#

Hi

timber osprey
#

what language should i learn after learning python

#

if i don't want any front end website

boreal umbra
#

None. You have won programming by learning Python.

timber osprey
#

or website related jobs

#

should i add some c++

#

to strengthen my foundation

flat gazelle
#

C++ is interesting, but learning C++ takes about a lifetime

timber osprey
#

ooo i see

flat gazelle
#

also, go to off topic probably

boreal umbra
#

This particular channel is for talking about the Python language itself

timber osprey
#

what is c++ mainly used for?

#

oww

#

sorry guys

#

didn't mean it

boreal umbra
#

Also I still crave validation for my proposed import syntax, but feel free to trash talk it as well.

flat gazelle
#

it would be nice sugar, but I see pretty much no advantage over

from typing import Dict, List
import typing as t
``` Though python could use a bit more terseness, it is quite verbose
boreal umbra
#

have you used Java?

flat gazelle
#

yes

boreal umbra
#

In that light I'm not sure I agree with that assessment about Python's verbosity.

peak spoke
#

I'm not sure if introducing the new syntax would be worth it as it's not as explicit, and importing like that is rarely done

boreal umbra
#

I guess it's not explicit in the sense that a * isn't self-explanatory, so you'd have to look up what it's doing in that context.

#

though it's not importing anything that isn't explicitly stated.

flat gazelle
#

I mean, there are more verbose languages, but we do not call java terse just because COBOL is more verbose. Compared to haskell, ruby, raku, some LISPs, C#, D and JS in some cases even, python is quite inexpressive.

sacred tinsel
#
from typing import Dict, List; import typing as t

lemon_swag

boreal umbra
#

but I don't want thatttttt

#

@flat gazelle I see what you're saying

peak spoke
#

I'd also expect a * to do something different at first, with it being for unpacking or as the star wildcard on imports

sacred tinsel
#

do you find yourself importing the module and then something from the module, separately, often?

flat gazelle
#

ye, it is quite inconsistent in that, though it is quite intuitive

boreal umbra
#

often I want to import the items I plan to use most frequently by name

sacred tinsel
#

when I see it I usually just assumed it was 2 people working on the module together and they didnt notice one or the other was already done lol

boreal umbra
#

and then I still want the rest of the module

#

I would include from typing import Dict, List, * in the same proposal, which in this case would be the same as from typing import Dict, list; import typing

#

Perl is more terse than Python, but you also have to spend eleven years figuring out what all the shorthands mean πŸ˜„

sacred tinsel
#

I'm not sure how I'd feel about the star behaving differently from just from typing import *

#

since if you do that you'll already have the Dict and List available right

boreal umbra
#

in the syntax I'm thinking of, if you import something explicitly, followed by a star, the star doesn't implicitly import everything

#

it imports the module itself by its own name.

#

which would be confusing, yes

sacred tinsel
#

maybe we should just remove star imports and replace them with this behaviour lol

flat gazelle
#

honestly, in 99% of the cases, I either use qualified imports or unqulified imports, not both

boreal umbra
#

That gets support from me, but we'd have to wait until python 4

sacred tinsel
#

I don't recall mixing them intentionally either

boreal umbra
#

unless we decide that people who use wildcard imports don't deserve to have their code work

#

which I also support.

#

Anyway I guess I can go to python-ideas but I imagine unless I include lots of caveats, they'll assume I have no design sense and I'll be eaten alive.

undone hare
#

In 3.9 you won't have (in most cases) to import typing anyway lemon_swag

red solar
#

ooh πŸ™‚

undone hare
red solar
#

contributed by Guido

#

ok ig we're getting it lol

#

wait isn't 3.9 out already?

undone hare
#

Nope, there only are release candidates betas for now

red solar
#

oh ok thank god i have time for 3.10 then

undone hare
#

They are what's new pages for un-released versions too

hasty thistle
#

ooohhh

#

that seems neat

#

oh wow, they finally added lcm to the math module?

red solar
#

if c++ can do it, python can do it 🀷

undone hare
#

Well, if C can do it CPython can do it lemon_pleased

languid dagger
#

It's not that you couldn't do it before, it just wasn't part of the stdlib

boreal umbra
#

@undone hare so basically the builtins are getting __class_getitem__?

undone hare
#

Yes, among other things

#

!pep 585

fallen slateBOT
#
**PEP 585 - Type Hinting Generics In Standard Collections**
Status

Accepted

Python-Version

3.9

Created

03-Mar-2019

Type

Standards Track

undone hare
#

Here is the related PEP with the implementation details lemon_pleased

grand field
true hollow
#

In 3.9 you won't have (in most cases) to import typing anyway lemon_swag
@undone hare is 3.9 released?????

languid dagger
#

@grand field That would be better in #python-discussion or an off topic channel, not this one

#

@wary plover Please don't advertise your channel, just wait for help.

undone hare
#

@true hollow not yet, only the betas are available right now

true hollow
#

I'll consider building one of them

undone hare
#

All released and un-released version have a what's new page

#

Even 3.10 has one

#

I'd be surprised if they aren't published binaries for 3.9

radiant fulcrum
#

If we assign a attribute to a class

#

say:

#

class FooBar:
  def __init__(self):
    pass
  
  def foo(self):
    pass
  
  def bar(self):
    pass

#

How come we cant then remove these attributes from a instance of it using
delattr

neon tartan
#

I don't see attributes? Only methods?

radiant fulcrum
#

mhmm

#

yet we can assign the same methods with setattr and remove them like that

#

but then we loose the ability to reference self which annoying

#

It would be nice if we had the ability to modify methods like attributes instead of say doing it with a decorator

neon tartan
#

Oh interesting, hmm I'm not sure so you'll have to wait for someone more experienced

north root
#

yet we can assign the same methods with setattr and remove them like that
these will not be bound methods

radiant fulcrum
#

yeah, which im assuming is also why we cannot reference self which makes sense

peak spoke
#

You can bind it when assigning to the instance

#

You can't delete the methods because they don't exist in instances

#

the "nicer" way to bind them when you already have an instance

In [11]: from types import MethodType

In [12]: class MyClass: ...

In [13]: def func(self): return self

In [14]: inst = MyClass()

In [15]: inst.func = MethodType(func, inst)

In [16]: inst
Out[16]: <__main__.MyClass at 0x67ec148>

In [17]: inst.func()
Out[17]: <__main__.MyClass at 0x67ec148>
radiant fulcrum
#

oOoh I havent actually seen MethodTypes been used that much

#

that's intersting

rare scaffold
#

@grand field is the content good though?

tawny shore
#

Yoo

unkempt rock
#

How make a thumbnail in the embed

golden imp
#

does List become obsolete then?

flat gazelle
#

It will stay for backwards compatability, but I think yes

golden imp
#

type hints are a wild ride

neon tartan
#

Wait, what's changing for type hints?

golden imp
#

soon you can do list[int] instead of needing List[int]

neon tartan
#

Ah

#

That probably simplifies things

#

It's going to break current typehints on leetcode yaaaaaaaay

flat gazelle
#

It won't break anything I would think.

neon tartan
#

oh so you just don't have to import from Typing

flat gazelle
#

The old typing.List are not removed, though they may be depracated soon

peak spoke
#

The PEP mentioned them being deprecated after builtin generics are implemented, but won't raise DeprecationWarning since it's just typing

timid orbit
#

having it supported on builtins would be great, yeah

peak spoke
#

If you do keep using them from typing for whatever reason then your code will break in the future

swift imp
#

Good

#

Because having to do from Typing include * gets annoying as fuck

#

I don't even use mypy but I still like adding type hints for completeness

flat gazelle
#

They are a great documentation tool

#

Though I wish people made more type aliases

swift imp
#

You can put in whatever u want really

peak spoke
#

The builtins are great but I don't think it's that much of a bother as most large projects will have a considerable amount of imports anyway which can be skipped over

swift imp
#

Like if I want an arg to be a dataframe I just do pd.DataFrame

#

def func(df:pd.DataFrame)->pd.DataFrame:

#

Clear and concise

unkempt rock
#

url_list.insert(y, url_list[y] + 1)

#

is this a correct syntax?

wraith brook
#

hello is anyone experienced with django

true hollow
#

@wraith brook #web-development πŸ™‚
this channel is to discuss PEPs, language design, the future of this language, etc

wraith brook
#

yes i already was talking there sorry

true hollow
#

nah don't worry

rare turtle
#

If I were to turn a website's API into a library, would it be better to use requests or aiohttp?

sacred tinsel
#

depends whether you want it to be asynchronous or not

#

using requests, your library will be easier to use for synchronous applications, whereas using aiohttp will make your library appropriate for async applications

#

it'd probably be very difficult to create something that will "just work" for both

#

usually you have libraries that are either strictly sync, or strictly async

rare turtle
#

Fair enough, I get what you mean. Thank you!

sacred tinsel
#

yeah no worries - it may be worth looking into whether there already exists a wrapper for the API, and then you can do something that the existing wrapper doesn't do

#

e.g. "while library A already wraps this API, it uses requests and blocks, my implementation is asynchronous and therefore appropriate for async apps" would be a good selling point

rare turtle
#

Makes sense, thanks!

true hollow
#

async ftw

#

make all your functions be coroutines

rigid tundra
#

I have an odd question, when doing a loop with an iterator that is the element and not a numerical incrementor, are you able to change what is in the existing array using that element iterator? Also is there any way to check to see which location that the element iterator is inside of the data structure that you are looping through?

Element:

for i in list:
  // code

Regular:

for i in range(len(list)):
  // code
sacred tinsel
#

no it's not possible, if I understand your question correctly

hazy anchor
#

enumerate() is one option

sacred tinsel
#
>>> nums = [1, 2, 3]
>>> 
>>> for n in nums:
...     n = 0
... 
>>> nums
[1, 2, 3]
hazy anchor
#

but only for the second part, knowing where the element came from

sacred tinsel
#

the loop binds the value to a new name n

peak spoke
#

iterting will create a new reference, with which youΒ΄can't access the reference inside what you're iterating over without some provided methods

sacred tinsel
#

you can change the value of n, but it doesn't write to the list

#

similarly:

>>> def func(n):
...     n = 0
... 
>>> number = 1
>>> func(number)
>>> number
1
rigid tundra
#

Ok, I was wondering how much more magic python had with loops. The majority of my coding experience is with C and Java so coming to a language where looping can be done is different ways was very new. It also still blows my mind that there is no variable scope in methods

#

Thank you

flat gazelle
#

Java also has a foreach loop

narrow kettle
#

Java has different ways to return an enumerable too

spark magnet
#

@rigid tundra what do you mean, "no variable scope in methods"? Methods have local variables.

rigid tundra
#

!e

def scope():
  for i in range(1):
    x = 3
  return x
print(scope())
fallen slateBOT
#

You are not allowed to use that command here. Please use the #bot-commands channel instead.

flint fractal
#

@spark magnet are you actually like a teacher?

spark magnet
#

@flint fractal not formally πŸ™‚

flint fractal
#

alright

flat gazelle
#

those are not methods, but yes, control flow does not have its own scope

spark magnet
#

Blocks don't create a new scope, i guess you meant.

rigid tundra
#

@spark magnet Blocks?

spark magnet
#

the statements inside the "for" loop

flat gazelle
#
if 5:
    "this is a block"
spark magnet
#

i'm not sure they are actually called blocks. sometimes they are called clauses.

narrow kettle
#

those are not methods, but yes, control flow does not have its own scope
A VERY confusing thing when i started

spark magnet
#

some people will tell you, once you've learned one or two languages, other languages are easy because you just need to learn the new syntax. those people are wrong πŸ™‚

flint fractal
#

facts

#

new language, new stuff to learn

flat gazelle
#

it is called body in ast it seems

rigid tundra
#

So, humor me on what exactly I said incorrectly

narrow kettle
#

I mean once you know one Lang it’s relatively easy to transfer concepts

flat gazelle
#

ah, I can tell what you meant. Within a single method, there is no variable scope

#

which is not quite true either

spark magnet
#

A method is a function defined within a class. They have their own scope, local variables ,like any function.

narrow kettle
#

A bit harder going from paradigm to paradigm

flat gazelle
#
def fun():
    a = 5
    def inner():
        a = 8
    inner()
    return a
print(fun()) # 5
wide shuttle
#

One think that could help is realizing that a "method" defined within the namespace of a class is just a regular function

#

Basically, this does two things:

def foo():
    pass
#

Python will make sure you've got a function object in memory and you assign a name to that function object

#

While variables work a bit differently in R (copy on assignment), R does show nicely what happens:

#
name <- function(a) {
    print(a)
}
cinder fern
#

hello. im learning python since few days and i couldnt understand how to use global variables. what am i doing wrong here?

def func1(a):
global b
b = a * 2
return b
def func2():
global b
c = b * 2
print(c)

narrow kettle
#

You didn’t declare a variable

wide shuttle
#

So,

def foo():
    pass

and

name = "hello"

work very similarly

#

If you then move that same logic to a class definition, you see what happens:

#
class MyClass:
    name = "hello"
    
    def foo():
        pass
#

You're creating objects as usual (a str object and a function object); it's just that the names here are class attributes

#

So, MyClass.foo will be assigned to the function object in memory and MyClass.name to the str object

#

The function object is just a normal function object

#

That's why you could also do:

#

!e

def my_odd_method(self):
    print(self)


class MyClass:
    method = my_odd_method


obj = MyClass()
obj.method()
fallen slateBOT
#

@wide shuttle :white_check_mark: Your eval job has completed with return code 0.

<__main__.MyClass object at 0x7fa03da3f9d0>
wide shuttle
#

There's nothing special about the function objects you define within a class namespace

#

The "magic" of inserting the instance (for self) happens when you access the class attribute on the instance of the class

rigid tundra
#

So, again humor me please. I would like to go back to what I said initially. I said "no variable scope in methods", wherein I meant that you can create a variable where ever you want inside of a method and it exists. As @flat gazelle pointed out, this becomes invalid when I created another method within a method which directly proves my statement incorrect. How can I tailor my statement to be correct? Would it be only that there is not scope within "blocks" as @spark magnet said?

flat gazelle
#

ye, I would say blocks do not create scope. Also method != function in python

#

as per Ves

rigid tundra
#

@wide shuttle Also thank you for the little lesson on objects

wide shuttle
#

"methods" are just regular function objects, so local names will go out of scope

#

However, we typically mutate the object the method is "acting on"

#

Like self.attribute = something

languid dagger
#

You need to use nonlocal same as with global for nested functions

rigid tundra
#

I learned something today πŸ™‚

spark magnet
#

@rigid tundra i see, it was the ambiguity of "in" that tripped us up.

#

"There are no new scopes created within methods other than the method scope itself"

rigid tundra
#

@spark magnet Yea, I thought that I had said in it in a correct way and I was confused when you guys said it was wrong, but I understand how I phrased it incorrectly

spark magnet
#

it wasn't incorrect it turns out. it's just hard to be super-precise.

flat gazelle
#

phrasing is hard, but you had the right idea, and that is what matters

rigid tundra
#

Wait wait wait.... so even though it was ambiguous I'm technically right with my phrase? I thought that because of the ambiguity of the use of "in" that it was incorrect because you can declare another method inside of the method and still have scope, and the way that I phrased it I said "no variable scope in methods" is false with that example mentioned.

flat gazelle
#

that is kind of the sole exception. But it is clear that the statement does not consider that as the claims it makes cannot be true in the case of nested methods

dry oar
#

cant you add variables to input texts?

rigid tundra
#

@flat gazelle Thank you for the clarification, my biggest fear is always looking like an idiot because I didn't get the proper diction or phrasing correctly πŸ˜‰

flat gazelle
#

if someone calls you dumb for a slightly incorrect statement, they are the dumb one. Now knowing something is not shameful

rigid tundra
#

So unfortunately I was never able to take a course in python directly, I had many projects through courses especially in my last year that got me to pick up python, and I feel decent at programming in it. However, I definitely lack the understanding of the language complexities such as scope, and immediately before that loop indexing. Does anyone have any recommendations as to where I may be able to find a good overview of very language specific things that python has that other languages do not?

wide shuttle
#

There are a couple of great videos on YouTube that could help you with getting familiar with some of the core mechanisms in Python

#

In fact, nedbat gave a couple of really good talks on PyCon

#

They explain things like how assignment works in Python (the relationship between names and values) and looping (that talk uses Python 2, but it's still great)

#

Other than that, there are great books like Fluent Python (the second edition will be released on Feb 9, 2021)

rigid tundra
#

@wide shuttle Awesome! I will be taking a look at these. @wide shuttle @flat gazelle @spark magnet @sacred tinsel Thank you for your time

smoky torrent
#

guys, i am learning python as my first coding language, and really have a hard time remembering all the
different commands and so on
any advice ? did you have this problem aswell?

flat gazelle
#

I suggest just writing them down in one place for easy lookup. With practice, they stick

#

that is what I do when I am using a language I am not all that used to

#

generally, you want to google something python cheat sheet

spark magnet
#

@smoky torrent btw, it's better not to start the same discussion in two channels at once

smoky torrent
#

alright, wont do it again:)

unkempt rock
#

how do i normalize number between 0-20

subtle pike
#
>>> xs = [*range(200)]
>>> min_x = min(xs)
>>> max_x = max(xs)
>>> xs_norm = [20 * (x - min_x) / (max_x - min_x) for x in xs]
>>> xs_norm[:3]
[0.0, 0.10050251256281408, 0.20100502512562815]
>>> xs_norm[-3:]
[19.798994974874372, 19.899497487437184, 20.0]

@unkempt rock

#

that is a statistics question, not a python language related question

#

preferably, use the appropriate channel

daring siren
#

Anyone aware of the speed of len()? Is it true that it's just O(1)?

wide shuttle
#

There's no single time complexity for the len function, as it depends on how it's implemented for the type

#

For most built-in types, the len just gets read off of a field in the C struct, which does make it fast

daring siren
#

Well, the most common implementation is Cpython, right?

wide shuttle
#

but, since you can implement support for len using the __len__ dunder in your own user-defined classes, it could be anything

brave badger
daring siren
#

Well I'll need to run it on strings, so I'm guessing I'm fine

wide shuttle
#

I think that page only lists it for list, @brave badger

brave badger
#

Yep

wide shuttle
#

For strings, it's the same

#

It just reads it directly from a field in the C struct

peak spoke
#

strings should keep that info since they're constant size

wide shuttle
#

but, for custom user-defined classes, you could write your own crappy implementation

peak spoke
#

I believe most, if not all, builtin sequences should keep it around since it's a lot cheaper to change on changes than counting when accessed

wide shuttle
#

They should probably do that anyway, to keep track of the memory allocation

#

You could always cheat, of course

from collections import UserList


class MyList(UserList):
    def __len__(self):
        count = 0
        for element in self:
            count += 1
        return count
daring siren
#

It's fine, I won't do any of that

#

Thanks guys!

flat gazelle
#

linked list have O(n) length query. And are one of the few data structures where n can actually be infinite

silent spear
#

Can someone explain to me what objects are

#

I already understand what classes are but I still don't understand objects are

visual shadow
#

Objects are the "realisation" of classes

#

If "human" is a class, then "you" are one object of the human class.

#

A class is like a blueprint. An object is like an actual item made from that blueprint.

silent spear
#

Oh ok

#

Thx

wide shuttle
#

@livid geyser Could you show me an example of what you mean?

#

Because it would work if you do this:

#

!e

def my_strange_method(self):
    print(self)


class Foo:
    method = my_strange_method

f = Foo()
f.method()
fallen slateBOT
#

@wide shuttle :white_check_mark: Your eval job has completed with return code 0.

<__main__.Foo object at 0x7fba609bf9d0>
livid geyser
#

If I

class Test:
  pass
def myfunc(self, *args):
  pass
Test.myfunc = myfunc```
Test.myfunc doesn't get self
#

Wait what

#

I swear I've literally done that and it doesn't bind

#

Wait no it's bound in the class

wide shuttle
#

Right, when you access the class attribute on the class object directly, you get the function object back as-is

livid geyser
#

!e

class Test:
  pass
def myfunc(self, *args):
  print(self)
Test.myfunc = myfunc
Test().myfunc()
fallen slateBOT
#

You are not allowed to use that command here. Please use the #bot-commands channel instead.

wide shuttle
#

but that holds for "normal" methods as well

livid geyser
#

oh

wide shuttle
#

!e

class Test:
  pass
def myfunc(self, *args):
  print(self)
Test.myfunc = myfunc
Test().myfunc()
fallen slateBOT
#

@wide shuttle :white_check_mark: Your eval job has completed with return code 0.

<__main__.Test object at 0x7ff72d6e69d0>
wide shuttle
#

That works fine

livid geyser
#

Wait what

wide shuttle
#

However, if you were to do Test.myfunc(), it wouldn't work

livid geyser
#

I have literally done that in my code before and it behaved weirdly

#

I'm so confused lol

wide shuttle
#

!e

class Test:
    def name(self):
        pass

print(Test.name)
fallen slateBOT
#

@wide shuttle :white_check_mark: Your eval job has completed with return code 0.

<function Test.name at 0x7f462243de50>
wide shuttle
#

If you access the class attribute on the class object (instead of on an instance), you get the function itself back

livid geyser
#

Did that get changed at some point or am I going mad lol

wide shuttle
#

It won't "bind" an instance, since there is no instance to bind

#

However, if you access the class attribute on an instance, you do get the binding

#

!e

class Test:
    def name(self):
        pass

t = Test()
print(t.name)
fallen slateBOT
#

@wide shuttle :white_check_mark: Your eval job has completed with return code 0.

<bound method Test.name of <__main__.Test object at 0x7f11e95f19d0>>
wide shuttle
#

As you can see, we're no longer getting a function object back, but a bound method

livid geyser
#

Yeah I know about that much

#

Maybe when I did it I messed up and bound it to the instance instead of the class

wide shuttle
#

Did you assign an instance attribute to the function object?

livid geyser
#

(N.B. by bound I mean set attr)

#

I probably did tbth

#

My bad lmao sorry for the wasted time

#

Hope your dinner was nice lol

wide shuttle
#

Still busy cooking

livid geyser
#

Oh ok lol

wide shuttle
#

although I don't really have to do anything atm

crimson dragon
#

Can I export python file with "pygal" package to browser without html and etc

unkempt rock
#

i cannot use pyaudio on my pc its showing error download microsoft visual studio but i have downloaded new version

radiant fulcrum
#
  1. This channel isnt really for help, 2) its not VS you need its c++ build tools if you read the error carefully
unkempt rock
#

Oooh c++ stuff πŸ™‚

pure seal
#

Hello

cloud crypt
#

Damn

#

memoryview is actually hella cool