#internals-and-peps

1 messages · Page 146 of 1

verbal escarp
#

but yeah, i'd prefer docker or venv for a controlled environment

halcyon trail
#

I'm pretty stoked. Getting CPU parallelism from threads would be really convenient.

#

multiprocessing isn't bad but there's a whole bunch of minor headaches

red solar
#

speaking of...

# 
from atomics import AtomicIntViewContext


if __name__ == "__main__":
    buf = bytearray(4)
    with AtomicIntViewContext(buffer=buf) as a:
        #  type(a) -> AtomicIntView
        print(a.exchange(5))
        print(a.fetch_neg())
        print(a.load())
    # print(a.load()) -> exception

better? @grave jolt (and ig @halcyon trail )

spark magnet
red solar
spark magnet
#

just incrementing the same int from two threads would lose updates.

red solar
#

oh yeah so most things aren't atomic in the first place, forgot lol

# 
i.__iadd__(1)

and here was me thinking this would be a single bytecode :/

spark magnet
#

that method call is one bytecode, but that's not how you increment an int.

#

"i += 1" is four bytecodes

red solar
#

Hmm, godbolt turned it into load and store for the method call

#

https://stackoverflow.com/a/18568017 i can definitely see people doing this as a hack though, and then getting their code broken if the GIL disappears (which isn’t a bad thing imo, but still)

halcyon trail
#

Seems fine to me doodspav

#

I probably wouldn't bother with context in the name personally, though that's taste

red solar
unkempt rock
#

Wrong channel my bad*

terse orchid
#

I have a private python package, hidden. In that package, I'd like to use another private package of mine, zempy.

In hidden's requirements, there is a line [-e ]git+https://gitlab.com/group/zempy.git@0.4.7#egg=zempy

And I'm installing hidden with python3 -m pip install [-e ]. (in hidden's main directory)

But I got an error:

error in hiddenco setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers; Parse error at "'-e git+h'": Expected W:(abcd...)

This seemed like working before. Suddenly it stopped working. Anyone has an idea? I have tried nearly everything in that requirements line. It's installing without that line. But I need zempy in hidden.

I have double checked install_requires parameter in setup. It is a list of string.

halcyon trail
#

or not as important, at the very least

flat gazelle
#

tbh, I would've probably made this API like

with buf as view:
    # stuff
```but I can see why you wouldn't like that
halcyon trail
#

he doesn't control the buffer type

#

so it cannot work that way

flat gazelle
#

ah, shame.

halcyon trail
#

the whole point of this API is that it provides an atomic view of memory that could come from wherever

#

Not really, if you don't do it that way then you are tied to a particular way of getting the memory

#

if you separate the responsibility for acquiring the buffer and for performing operations on it atomically then you can do atomic operations on a buffer that comes from anywhere without writing any more code

red solar
#

although type hinting is a separate issue that i should probably solve, because rn i don't have any sort of base class that i expose :/

halcyon trail
#

well, this thing only lives purely inside the context manager, so it's a bit unusual to pass it into other stuff, though not non-existent I suppose. But yeah, fair enough, i don't think context would be my first choice but I dont' know immediately what is better

red solar
#

Manager feels wrong, and Factory i didn't like because it's single use

halcyon trail
#

Is the outer thing even a class? it can just be a function, can't it?

#

the way that e.g. open is a function

red solar
#

it's a class that implements __enter__ and __exit__

halcyon trail
#

i forget the chain of reasoning for why two classes are needed in the first place, tbh

red solar
#

one implements __enter__ and __exit__, and the other implements all the operations

#

(kinda)

halcyon trail
#

oh, because you only want the useful API on the inner thing

red solar
#

yeah

halcyon trail
#

and enter and exit on the outer thing

red solar
#

(AtomicCore isn't exposed, so creating an AtomicBaseView object isn't easy for the user)

halcyon trail
#

okay, but the enter/exit class is never used by the user, should never be type hinted, or anything

#

So instead of the user calling the init function for this class that they never really use otherwise, they could just call a function instead

# 
with view_atomically(buffer) as :
    ...
red solar
#

hmm

halcyon trail
#

and now the name of the "outer" class is an implementation detail, there's just the view_atomically function and the inner type

#

which you can call AtomicView

red solar
#

ok i like it 🙂

#

but i need to figure out where in that interface the user specifies if it's Int, Uint, or Bytes

#

brb shopping

red solar
#

view_atomically(buffer, as=AtomicIntView)?

#

hmm

#

or maybe make a new enum just for that

halcyon trail
#

err why not just pass the type

#

seems more logical

#

i guess with an enum you constrain it

#

so maybe that's okay to, if you want it to work in terms of type annotations though it's simpler to just pass the type

grave jolt
red solar
#

damn :/

#

would've been nice

surreal sun
#

I wonder if as could be controlled with a dunder method
sounds cursed but cool nevertheless

grave jolt
#

I think we need less dunders, operators and other syntax (like walrus, f-strings etc.)

#

not sure about patma yet

red solar
#

i wouldn't say less fstrings

noble kelp
#

not specifically python related, but is it possible to hit redis with so much traffic at once that you essentially DOS the eviction process? hit OOM and saw 0 evictions from a hard spike in traffic--anybody seen that before?

halcyon trail
grave jolt
halcyon trail
#

f-strings however are amazing and are really the right way to do strings, not sure why you don't like them (and I think they're the future for more languages as well)

#

Ah, so you are talking about operators that are not yet standardized

grave jolt
#

as in, I think adding new syntax for what's already possible isn't a good idea

#

most of the time

halcyon trail
#

sorry that wasn't clear since you said "less", thought you were talking about stuff that's already in

red solar
halcyon trail
#

(and walrus, f-strings are already in)

#

Maybe, I think it's hard to say, it does feel like python has gone surprisingly aggressive on adding new syntax at times.

#

walrus in particular (and I know adding walrus was very controversial, both generally and in the steering council)

#

function composition I'm sympathetic to in principle, but i'm just not sure it suits the language, the language can''t have real pretenses of being functional with such weak lambdas

grave jolt
#

his palms are sweaty, lambdas weak, arms are heavy...

halcyon trail
#

There's vomit on his editor already, code spaghetti

grave jolt
#

He's nervous, but on the surface he looks clean and tested

halcyon trail
#

He's nervous, but on the surface he's webscale-ready
To drop async, but he keeps on forgetting
What he queued down, the server goes so loud,
The server runs but the requests won't come out!

grave jolt
#

!otn a the-requests-won't-come-out

fallen slateBOT
#

:ok_hand: Added the-requests-won’t-come-out to the names list.

halcyon trail
#

lol what

#

what does that mean

grave jolt
#

where do you think names for off-topic channels come from

halcyon trail
#

i have wondered that before

grave jolt
#

we have 1402 names in rotation so far

halcyon trail
#

nice

grave jolt
#

I think we're adding them faster than they appear tho

queen dust
#

wait how often are you rotating ot names?

surreal sun
#

Every day at 12 AM UTC

grave jolt
surreal sun
queen dust
peak spoke
#

old for the most part

grave jolt
peak spoke
#

and I guess with the lesser used modules it's mostly about what the module's maintainer let go through as it's usually just one person

halcyon trail
surreal sun
halcyon trail
#

The people who write it are not, IME, especially amazing coders. They're not bad coders either. But it really just boils down to who wants to do it, had an excuse to do it at work, etc etc.

#

Let me rephrase; they could be amazing coders, but they aren't necessarily

#

There was a point where I would have assumed that every line in the python standard library was pure gold but at this point I would not take it as a given

queen dust
halcyon trail
#

what's k8s?

queen dust
halcyon trail
#

ah

grave jolt
halcyon trail
#

well it's in Go so I don't need to look at it to know that 😉

grave jolt
halcyon trail
#

half joking but I read about some of the hackery they needed to do because they wanted some kind of type registry and go's lack of generics makes a lot of this kind of stuff a headache

surreal sun
queen dust
peak spoke
#

From what I've found it's usually knowledgeable people in one area but then they cover more things and for the other things it's subpar; and some oss people insist on very weird things in regards on how things should be written or the code style - then if they're a maintainer it propagates through that

queen dust
halcyon trail
#

@grave jolt I've had many such moments over the years. I remember years ago as I was starting to get really good at C++, and the zero MQ guy had a bunch of blog posts on the problems with C++, and the posts are just objectively bad. C++ people love trashing C++ but the post was just all over the place.

#

And i was just confused, like, this guy wrote ZMQ which is used so many places, how can he give an argument on RAII or intrusive data structures that shows he doesn't understand them at all 🤷‍♂️

surreal sun
#

Most of the "[language] bad" articles I find can either be horribly written or just don't give any constructive criticism whatsoever

grave jolt
peak spoke
#

my latest experience like that was a guy insisting on still supporting python 2 in a (small) codebase which was a mess and could really use a refactor; apparently instead of improving the project and letting python 2 support be through the older versions of the project, they'd rather let it stagnate

queen dust
halcyon trail
#

My overall conclusion is that if you work at a place where people are well paid, the average level of a developer at your work is probably better than the average level in OSS

#

It's not really that surprising; most of the best developers I ever met didn't have any interest in maintaining any kind of non-trivial OSS project.

#

it's so much work

queen dust
#

one of my company's source got leaked and people were surprised at how sketchy some of it was lemon_angrysad

halcyon trail
#

lol nice

#

yeah I mean I guess in the end my sample size is not very big

queen dust
queen dust
halcyon trail
#

the arguments don't really matter, everyone knows PHP sucks by now, the writing style is just funny in places

surreal sun
halcyon trail
#

yeah, that's when they're written by people who don't know the language well, so they focus on superficial things

#

python guy learns C++ for a week - "C++ bad cause semi-colons"

#

to get a good article like that you have to have someone who thinks the language is flawed but has really been forced to write enough code in it, over the course of years, to understand it really well

#

and then still able to talk about it fairly objectively

#

it's super rare

#

there was a very good article IMHO on Scala's warts a while ago

surreal sun
surreal sun
halcyon trail
halcyon trail
#

Like, posts like this leave no room for satirical medium posts

surreal sun
halcyon trail
#

How could I write a blog post satirizing medium when stuff like this exists? Nobody would understand its satire.

surreal sun
#

I don't necessarily like dynamic typing, but to call someone who disagrees with you incompetent is uh

surreal sun
halcyon trail
unborn viper
#

Imagine living in a world without dynamic typing languages tho, saying that it is bad it's like saying a toy car it's not suitable for a kid, which means the kid should learn driving with a SUV.

halcyon trail
#

wait, but now you're saying that dynamically typed languages are toys?

#

I also don't really agree that it's necessarily better to learn with dynamically typed languages than statically typed languages. But there's not much real empirical evidence either way.

unborn viper
#

They serve a purpose, saying that they are not ok, means that they do not serve their purpose anymore. Purpose for me, it's easy prototyping and readability

#

I personally started learning code with C, but for most of my beginner projects, i regret not using Python earlier just for the sake of having a working code in much less time.

grave jolt
#

C is not a good example of a static language...

unborn viper
#

But whatever, non constructive criticism is actually the worst, in my honest opinion

unborn viper
queen dust
grave jolt
queen dust
#

it's statically typed but (this is a subjective term) weakly typed because of type coercion/implicit conversions that more robustly typed languages like java and c++ don't allow

unborn viper
queen dust
halcyon trail
#

void* casts are needed in C for extremely basic and common things

#

callbacks for example

#

which is awful

grave jolt
halcyon trail
#

LOL

#

the bot was written in C and had some UB

#

Haskell seems like a rough start, and gradually typed to me seems more confusing

#

(to start)

grave jolt
halcyon trail
#

I would probably go with one of the bog-standard languages; Java, C#, maybe Kotlin

#

maybe not Java just because of the excessive OO focus

grave jolt
halcyon trail
#

sure, fair enough

queen dust
#

i mean even null can be considered sus when it comes to typing ig

grave jolt
#

null is very sus

halcyon trail
#

null's sus-ness confirmed

grave jolt
#

hey we finally agree on something @halcyon trail

halcyon trail
#

@grave jolt it's hebrew 🙂

#

oh i thought we agree fairly often?

queen dust
grave jolt
halcyon trail
#

that was the standard intro class at MIT for decades

#

not that surprising

grave jolt
#

hm right

halcyon trail
#

i'm probably biased but I think lisps are much easier to learn than haskell

queen dust
#

it seems like a good intro language class

queen dust
halcyon trail
#

lisps have weird syntax but that is pretty shallow. in terms of how the actual language works, until you start learning about macros and the fancier features enabled by homoiconicity, lisps can feel very bog standard

grave jolt
#

I guess Haskell would be way harder for a beginner to get stuff done with. Like, imagine writing a Discord bot in Haskell.

halcyon trail
#

haskell you need to learn about purity, monads, from relatively early on

queen dust
halcyon trail
#

I can imagine 🙂

grave jolt
#

from now on I'm going to call :: paamayim nekudotayim

manic sundial
halcyon trail
#

it's hebrew for "twice two-dots" basically

#

I'm not actually sure that nekudotayim is a real word

#

my hebrew is like a first grade level or so

queen dust
halcyon trail
#

but Hebrew as language, has singular, plural, and a special "two" suffix

#

but it can't be used with all nouns, only certain nouns

#

time durations, "things that come in pairs". Like: pants, bicycles, glasses, etc all have this "two" suffix.

unkempt rock
#

I GOT CRAZY

#

I GOT CRAZYZZZZ

#

SOMEONE HELP ME

halcyon trail
#

obviously though it's insane that random bits of hebrew are coming out of error messages

unkempt rock
#

I GOT CRAZY OH GOOOD

grave jolt
#

@unkempt rock do you need help?

queen dust
halcyon trail
#

PHP

unkempt rock
#

YES I GOT TOO CRAZY AND I CANT MORE EAT OR DRINK

halcyon trail
#

PHP uses paamayim nekudotayim in user facing error messages

#

which is a way of referencing :: in Hebrew

queen dust
grave jolt
halcyon trail
#

clearly

queen dust
#

templeOS should've used PHP-based shell?

halcyon trail
#

it's pretty embarrassing to me for how many people that's the main source of hebrew exposure 😛

astral gazelle
grave jolt
#

@unkempt rock can you stop spamming please?

unkempt rock
#

i love the funny thing

astral gazelle
#

Go be funny elsewhere

grave jolt
#

Indeed, you are an award-winning comedian. But this is not the channel for this.

unkempt rock
#

i need to be funny in python server

#

i became fuuny with muchs class

#

someone wanna be funny with me?

queen dust
#

knock knock

unkempt rock
#

please lets be funny

#

😎

#

it is a funny person ⬆️

astral gazelle
#

Are we gonna have to ping mods or are you going to take the hint that youre offtopic

unkempt rock
astral gazelle
#

<@&831776746206265384>

unkempt rock
undone hare
#

Indeed, this isn't a place to really funny

#

There are other channels for that

unkempt rock
#

i do not like moderators because they arent funny

#

oh but only if u call me funny

undone hare
#

!mute @unkempt rock 5h Sounds like you need to take a break. Feel free to come back when you have calmed down and stop calling someone unfunny or whatever, please stick to the channel topic.

fallen slateBOT
#

:incoming_envelope: :ok_hand: applied mute to @unkempt rock until <t:1636072404:f> (4 hours and 59 minutes).

verbal escarp
#

at least that's what i would expect nowadays with such a project, but back then there probably wasn't even proper tools for versioning etc

red solar
# 
buf = bytearray(8)
with view_atomically(buffer=8).as_int() as a:
    x = a.load()

is this too much as?

spice pecan
#

I think as_int could really just be a kwarg

#

as_type=int or something along those lines perhaps

red solar
#

the issue with that is the types i would need there are huge :/

# 
buf = bytearray(8)
with view_atomically(buffer=buf, as_type=AtomicIntView) as a:
    x = a.load()
#

the other thing is idk how this works with type hinting now, will linters and IDEs correctly deduce that a has type AtomicIntView?

spice pecan
#

You could have a type mapping

red solar
#

what do i map for AtomicUintView?

spice pecan
#

So you'd pass 'int' or int as the arg, which would then internally be translated to its respective atomic class

#

Though as_<type>() is still fairly okay, tbf

red solar
#

ideally i'd have a good name for the function such that i can have 3 functions rather than 1 function with 3 possible as_type params

#

but i can't think of a good name :/

spice pecan
#

I don't remember exactly, what was wrong with exposing Atomic<type>View?

red solar
little ginkgo
#

Good afternoon, where can i read about packaging scripts and orchestrating them? For instance, I have 4 scripts that retrieve various data from APIs, and 4 scripts that convert JSON to CSV... what's the easiest way to create a shell to execute them? (should I make them into classes or modules? thank you)

red solar
#

so AtomicViewContext returns AtomicView

red solar
little ginkgo
#

no, more like organizing them

#

so that instead of 8 files, I maybe put everything at one

#

and then of course, logging / restarting /etc

red solar
#

hmm... what's wrong with having 8 files?

#

not sure why you need 4 scripts for json to csv tho, that sounds like something you could do in a single class

little ginkgo
#

each has a different structure and nesting levels

red solar
#

hmm

#

well the csv thing can be a package with 4 modules in it, each with its own class

#

not sure about the api yet

halcyon trail
red solar
halcyon trail
#

I don't really see an issue either with view_atomically(buf, AtomicIntView)

#

you don't have to use keyword arguments here, there's only two arguments and they're pretty clearly two different things

#

you could also just have different functions, assuming this is basically always going to be hardcoded, or at least offer that for the majority case

red solar
#

hmm... if i don't have to use kwargs that looks nicer

#

if you can think of a good name for the different functions, that would be ideal but probably the hardest

halcyon trail
# 
view_int_atomically
view_foo_atomically
#

etc

#

basically just sticking the argument into the name lol but shorter

red solar
#

the issue is that i don't want the name to make the user think that the first param has to be an int or bytes

#

rn i'm fixing the inheritance stuff tho 🙂 easier problem

#

if i have Atomic, AtomicIntegral, and AtomicInt/Uint, me exposing all 3 is fine right? even though the first two are useless own, and are just there for type hinting?

spice pecan
#

atomically_view_as_int?

red solar
#

ehhh idk it still feels iffy, atomically_view_as_int(...) as atomic_int:?

spice pecan
red solar
#

abstract? how do you do this in Python?

spice pecan
#

!d abc

fallen slateBOT
#
abc

Source code: Lib/abc.py

This module provides the infrastructure for defining abstract base classes (ABCs) in Python, as outlined in PEP 3119; see the PEP for why this was added to Python. (See also PEP 3141 and the numbers module regarding a type hierarchy for numbers based on ABCs.)

The collections module has some concrete classes that derive from ABCs; these can, of course, be further derived. In addition, the collections.abc submodule has some ABCs that can be used to test whether a class or instance provides a particular interface, for example, if it is hashable or if it is a mapping.

This module provides the metaclass ABCMeta for defining ABCs and a helper class ABC to alternatively define ABCs through inheritance:

Similar names: aiohttp.doc.abc
red solar
#

oh hmmm

#

you made me realise the base classes still need to have functions for them to be useful to linters, they can't just be empty :/

spice pecan
#

Yeah

#

Making them either ABCs or Protocols could be a good idea

spice pecan
red solar
#

hypothetical question, but if i (in the future) rewrote this whole thing in C, could i just have a type stub for the abstract classes and not implement them in C? (not 100% sure if that question makes sense)

red solar
halcyon trail
#

if they're useless on their own why would you use them in type hinting though?

#

I mean you might as the library maintainer, but users wouldn't

spice pecan
#

Depends one what "useless" means in this context. If they're just interfaces/base classes for actually functional classes, having them public would indeed be useful, though that falls into the ABC/protocol category

red solar
#

like what operations i want in Atomic, in AtomicIntegral, and in AtomicInt

red solar
#

Atomic just implements __del__ and __init__

#

the mixins rely on a single attribute available from Atomic, so it's just a matter of rejiggling which stage stuff is distributed in

halcyon trail
#

I wouldn't spend too much time worrying about how to construct a complex inheritance hieararchy tbh

dusk comet
# 
>>> class X:
...   def __bytes__(self):
...     print('__bytes__ called')
...     return b''
...
>>> bytes(X())
__bytes__ called
b''
>>> bytearray(bytes(X()))
__bytes__ called
bytearray(b'')
>>> bytearray(X())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: cannot convert 'X' object to bytearray
```why `__bytes__` is not called on `bytearray(...)` ?
how to implement custom conversion to bytearray?
red solar
#

__index__ might work even though that's not what it's meant for

#

(assuming the interpreter doesn't check that you return an int)

#

nvm you can't do that

dusk comet
#

ok, thanks...

red solar
#

(not so much worried about stuff off the bat, more about any gotchas down the road)

#

ah nvm seems python deals with diamond inheritance stuff fine 🙂

grave jolt
# 
       object
         ^^
        /  \
       /    \
      /      \
     /        \
    ^          ^
    A          B
    ^          ^
     \        /
      \      /
       \    /
        \  /
         ^^
         C
raven ridge
#

and whether or not it works fine mostly depends on whether you understand how super() works.

#

Python has cooperative multiple inheritance, and it's each class's job to cooperate.

red solar
#

Well it’s more none of the mixins have state or __init__ so I don’t need to cooperate there 🙂

torpid bridge
paper echo
raven ridge
#

that would imply that they can't have even a __init__

paper echo
#

right, only one of them should have a user-defined __init__

#

otherwise you rely on MRO and nobody wants that

#

(unless you explicitly know you want that)

swift imp
#

I get inheritance is tough, especially multiple

#

But like ive not once found myself trying to subclass from another library

#

Just doesn't seem like something people do. Only time I've subclassed is for classes I wrote and I specifically will make sure there's no conflicts

lusty scroll
raven ridge
#

even in Java, subclassing other people's classes is usually discouraged. "Prefer composition to inheritance" is a pretty general rule, it even appears in the Design Patterns book.

static bluff
#

Lemme ask you guys something

#

Why Python?

#

And if not Python, what?

static bluff
# swift imp I get inheritance is tough, especially multiple

I think the salient point with regard to subclassing other people's code is wether or not is was designed to be subclassed. I use a library called cefpython. Its a binding around the code that makes chrome run. There's a class inside it that you can't really access. Its somewhere in there, but you normally get an instance of it by called 'createBrowser' or 'createBrowserSync', and there's a bunch of excess overhead that needs to be processed when you make one of these objects, so just instantiating one even if you did find the class won't do you much good

#

On the other hand, there's wxPython. Its 'App' class designed to be subclassed. That's pretty much the whole point. You subclass it, that gives you access to the main routine of the program and all the different points of contact with it, and you specify the behaviour you need

deft pagoda
surreal sun
# static bluff Why Python?

Python has a lot of different libraries and it’s generally quite easy to write code and worry more about logic than syntax. Now for what else other than python, pythons the only language I’m decent at so

deft pagoda
#

no, it doesn't it just forbids the order in which the bases appear:

In [3]: class A:
   ...:     ...
   ...: 

In [4]: class B:
   ...:     ...
   ...: 

In [5]: class C(A):
   ...:     ...
   ...: 

In [6]: class D(A, B):
   ...:     ...
   ...: 

In [7]: class E(C, D):
   ...:     ...
   ...: 

In [8]: class E(C, A):
   ...:     ...
   ...: 

In [9]: class E(A, C):
   ...:     ...
   ...: 
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-9-23a77786bae9> in <module>
----> 1 class E(A, C):
      2     ...
      3

TypeError: Cannot create a consistent method resolution
order (MRO) for bases A, C
languid yoke
manic sundial
still wolf
#

thanks ❤️

weary flax
#

when using selenium, wich method is faster: xpath or ID, class name ?

#

i just need to return a value, but every second is precious when executing the program

ruby pilot
#

shouldn't take more than a millisecond or two anyway

#

but you should really just benchmark them in your specific case

#

my money's on ID though

#

@weary flax mention cuz it's been 2 hours

paper echo
#

and the rule of thumb for performance optimization is: don't guess, benchmark

#

chances are that isn't the bottleneck in your program anyway, unless you are sure (and have verified) that i/o isn't the bottleneck

red solar
#

cProfile ftw 🙂

still wolf
#

@red solarwhat is that Patreon one role?

red solar
still wolf
#

lol waiting for you

#

oo

#

thanks

red solar
#

not sure who pays extra for the dark colour when the light one's so much nicer

still wolf
#

@red solarmeans you pays a amount every month?

#

maybe $5 right?

red solar
#

I guess - I haven’t paid much attention to it

still wolf
paper echo
#

@unkempt rock thanks for the pointer #help-pie message - by "integer conversion" i just meant the d format string as opposed to e.g. s

unkempt rock
#

oh. didn't realise that was even a thing.

#

as opposed to hex/bin I suppose

#

or when padding

paper echo
#

annoyingly this seems to be for "full" format strings

#

!e ```python
import string
for literal, name, spec, conv in string.Formatter().parse('{:d}'):
print((literal, name, spec, conv))

fallen slateBOT
#

@paper echo :white_check_mark: Your eval job has completed with return code 0.

('', '', 'd', None)
paper echo
#

i just want the code that parses the "spec" part... blah

unkempt rock
#

oh it's too useful for you? xd

paper echo
#

lol yes

#

i literally just want to implement __format__!

#

which is what format() calls

unkempt rock
#

i think you can afford the one line it takes to take it out of the tuple

#

or do you get a bunch of other stuff along with it as well?

#

I suppose you do.

peak spoke
#

In what way do you want to parse it?

paper echo
#

i'd have to do something like "{:" + fmt + "}" which seems really flaky

unkempt rock
#

f"{:{fmt}}"

paper echo
unkempt rock
#

oh maybe escape that.

paper echo
#

f"{{:{fmt}}}"

#

for now i'm just doing fmt.endswith('s') but i'm sure that's wrong

#

so instead i'm gonna go shave a yak and write a standalone format specification parser using lark or something

#

guess i could do it in regex too

#

looks like it's all wrapped up in c code and the internal formatter parser thing isn't exposed publicly

#

annoying!

unkempt rock
#

I can't even figure out where format is defined in this


    def format_field(self, value, format_spec):
        return format(value, format_spec)
#

there's a format in the class but that's... IN THE CLASS

paper echo
#

format is a builtin

#

!d format

fallen slateBOT
#
format 

format(value[, format_spec])```
Convert a *value* to a “formatted” representation, as controlled by *format\_spec*. The interpretation of *format\_spec* will depend on the type of the *value* argument; however, there is a standard formatting syntax that is used by most built-in types: [Format Specification Mini-Language](https://docs.python.org/3/library/string.html#formatspec).

The default *format\_spec* is an empty string which usually gives the same effect as calling [`str(value)`](https://docs.python.org/3/library/stdtypes.html#str "str").

A call to `format(value, format_spec)` is translated to `type(value).__format__(value, format_spec)` which bypasses the instance dictionary when searching for the value’s `__format__()` method. A [`TypeError`](https://docs.python.org/3/library/exceptions.html#TypeError "TypeError") exception is raised if the method search reaches [`object`](https://docs.python.org/3/library/functions.html#object "object") and the *format\_spec* is non-empty, or if either the *format\_spec* or the return value are not strings.
unkempt rock
#

oh. weird.

paper echo
fallen slateBOT
#

Objects/stringlib/unicode_format.h line 494

get_field_and_spec, and renders the field into the output string.```
paper echo
fallen slateBOT
#

@paper echo :white_check_mark: Your eval job has completed with return code 0.

0007b
unkempt rock
#

yeah so that's for a single value

paper echo
#

yep

unkempt rock
#

hmm

#

seems useless as a builtin

paper echo
#

why?

unkempt rock
#

who would ever format a single value

paper echo
#

i do it all the time

unkempt rock
#

I mean - str.format or f string would allow the same thing. but. kay.

paper echo
#

format predates the latter and i think maybe predates the former, or at least was introduced concurrently with it

unkempt rock
#

where is the builtins module defined?

# 
→ fd builtin
Doc/library/builtins.rst
Lib/_sitebuiltins.py
Lib/test/test_builtin.py
Lib/test/test_importlib/builtin
Misc/NEWS.d/next/Core and Builtins
paper echo
#

!e ```python
class Weird:
def format(self, fmt):
if fmt == 'z':
return '<<weird>>'
else:
raise TypeError(f'Unknown format code {fmt!r} for object of type {self.class.name!r}')

print(format(Weird(), 'z'))
print(format(Weird(), 's'))

fallen slateBOT
#

@paper echo :x: Your eval job has completed with return code 1.

001 | <<weird>>
002 | Traceback (most recent call last):
003 |   File "<string>", line 9, in <module>
004 |   File "<string>", line 6, in __format__
005 | TypeError: Unknown format code 's' for object of type 'Weird'
raven ridge
unkempt rock
#

love you

paper echo
#

!e ```python
class Weird:
def format(self, fmt):
if fmt == 'z':
return '<<weird>>'
else:
raise TypeError(f'Unknown format code {fmt!r} for object of type {self.class.name!r}')

print('{:z}'.format(Weird()))
print('{:s}'.format(Weird()))

fallen slateBOT
#

@paper echo :x: Your eval job has completed with return code 1.

001 | <<weird>>
002 | Traceback (most recent call last):
003 |   File "<string>", line 9, in <module>
004 |   File "<string>", line 6, in __format__
005 | TypeError: Unknown format code 's' for object of type 'Weird'
paper echo
#

so __format__ allows you to configure how format() and f"" and str.format work

#

the former probably being the least useful, yes, but still somewhat useful

unkempt rock
#

ohh. types receive the format string themselves.
..... so you can have custom format specifiers

raven ridge
#

Date/time types use that to support strftime codes, IIRC

paper echo
#

!e ```python
from datetime import datetime as DateTime
print(format(DateTime.now(), '%Y-%m-%d'))

fallen slateBOT
#

@paper echo :white_check_mark: Your eval job has completed with return code 0.

2021-11-05
paper echo
#

!e ```python
from datetime import datetime as DateTime
print(f'{DateTime.now():%Y-%m-%d}')

fallen slateBOT
#

@paper echo :white_check_mark: Your eval job has completed with return code 0.

2021-11-05
paper echo
#

WHAT

unkempt rock
#

how to confuse your collegues

paper echo
#

THIS IS AMAZING

unkempt rock
#

doesn't look bad though.

paper echo
#

so you basically can't/shouldn't ever rely on the format string because it could be arbitrary

unkempt rock
#

I think it skipped __format__ for empty/null specification

paper echo
#

yes that's just str()

#

in which case i think the only sensible solution to my case is

    def __format__(self, fmt) -> str:
        try:
            format(self.to_int(), fmt)
        except TypeError;
            format(str(self), fmt)
unkempt rock
#

yeah but it doesn't even receive "" which it conceivably could have

paper echo
#

!e ```python
from datetime import datetime as DateTime
print(f'{DateTime.now():%Y-%m-%d!r}')

fallen slateBOT
#

@paper echo :white_check_mark: Your eval job has completed with return code 0.

2021-11-05!r
paper echo
#

!e ```python
from datetime import datetime as DateTime
print(f'{DateTime.now()!r:%Y-%m-%d}')

fallen slateBOT
#

@paper echo :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 2, in <module>
003 | ValueError: Invalid format specifier
paper echo
#

!e ```python
from datetime import datetime as DateTime
print(f'{DateTime.now()!r}')

fallen slateBOT
#

@paper echo :white_check_mark: Your eval job has completed with return code 0.

datetime.datetime(2021, 11, 5, 20, 35, 33, 488011)
paper echo
#

i guess you can't combine !s/!r with :? i thought maybe you could somehow

#

but it makes sense that you can't

unkempt rock
#

{x!r:otherstuff}, not that it makes much sense

peak spoke
#

I was missing the ability to do !s!r recently

unkempt rock
#

I mean I guess that makes sense, lol

peak spoke
paper echo
unkempt rock
paper echo
drifting harbor
# 
print("hello")
paper echo
#

!e ```python
from datetime import datetime as DateTime
print(f'{DateTime.now()!r:s}')

fallen slateBOT
#

@paper echo :white_check_mark: Your eval job has completed with return code 0.

datetime.datetime(2021, 11, 5, 20, 37, 42, 473487)
paper echo
#

tldr yes but i forget how to specify padding

#

wait... you can put a replacement field as the format spec?

peak spoke
#

Could it be the !r applying before and then only supplying the string as as the string to format?

paper echo
#

!e ```python
from datetime import datetime as DateTime
fmt = '%Y-%m-%d'
print(f'{DateTime.now():{fmt}}')

fallen slateBOT
#

@paper echo :white_check_mark: Your eval job has completed with return code 0.

2021-11-05
paper echo
#

this is so cursed that the parser doesn't even know how to handle it

#

i guess it helps to read the grammar

peak spoke
verbal escarp
#

🍿

paper echo
#

!e ```python
from datetime import datetime as DateTime
fmt1 = '%Y-'
fmt2 = '%m-%d'
print(f'{DateTime.now():{fmt1}{fmt2}}')

fallen slateBOT
#

@paper echo :white_check_mark: Your eval job has completed with return code 0.

2021-11-05
paper echo
#

🤣 no way

# 
format_spec  ::=  (literal_char | NULL | replacement_field)*
grave jolt
fallen slateBOT
#

@grave jolt :white_check_mark: Your eval job has completed with return code 0.

'42'
grave jolt
#

😉

#

ez

verbal escarp
paper echo
#

i'm not silly, the grammar is silly

#

f'{str(x)!r}'

#

i'm actually a bit disappointed you can't do !s!r

peak spoke
#

similar weirdness I encountered is that you can chain ifs in a comprehension

#

e.g. [a for a in range(10) if a if a if a]

paper echo
#

what does that mean

verbal escarp
#

o.O

peak spoke
#

should be the same as anding them

paper echo
flat gazelle
#

I mean, IG if really is , from set builder notation

grave jolt
#

i-

#

that's entirely new

#

to me

paper echo
#

same! how does associativity work with the ifs? that might actually be useful for reducing nesting ()s

#

i assume the ifs are "outermost"

verbal escarp
#

wait what.. for ifs?

paper echo
#

!e ```python
[a for a in range(10) if a or False if a or False if a or False]

verbal escarp
#

is that only in comprehensions?

flat gazelle
#

inner first short circuiting as far as I can tell

fallen slateBOT
#

@paper echo :warning: Your eval job has completed with return code 0.

[No output]
flat gazelle
#

!e
[a for a in range(10) if print(2) if print(1)]

fallen slateBOT
#

@flat gazelle :white_check_mark: Your eval job has completed with return code 0.

001 | 2
002 | 2
003 | 2
004 | 2
005 | 2
006 | 2
007 | 2
008 | 2
009 | 2
010 | 2
verbal escarp
#

and which versions?

paper echo
#

listcomp, genexp, setcomp, dictcomp

verbal escarp
#

k

paper echo
#

i wonder why it's genexp and not gencomp

unkempt rock
# 
[ result
  # at least one for's and if's
]
verbal escarp
#

and there i thought my idea to have conditions in for-loops came true

unkempt rock
#

for some reason it isn't allowed to only have a single if

paper echo
#

so if a if b if c is like

if a:
    if b:
        if c:
#

which is the same as and

unkempt rock
# 
>>> [3 if True]
  File "<stdin>", line 1
    [3 if True]
     ^^^^^^^^^
SyntaxError: expected 'else' after 'if' expression

^ this needs to be fixed.

verbal escarp
#

sigh.. yep

paper echo
#

i guess it tries to parse the "named expression" first

peak spoke
verbal escarp
#

there's some AST manipulation POC to expand foo if x to foo if x else None

unkempt rock
verbal escarp
#

nobody thinks you're kidding

unkempt rock
#

idk usually nobody believes me when I say things like that xD

verbal escarp
surreal sun
paper echo
surreal sun
#

I would call it Datetime ngl

#

DateTime doesn't sit with me well, even though it makes more sense

paper echo
#

i'll settle for that, easier to type anyway

#

the capital T is a bit awkward

grave jolt
#

hmm

rich cradle
#

I like capital T since it's a new word

surreal sun
#

The naming conventions in the stdlib are quite inconsistent

paper echo
#

considering it's a 30 year old language they're actually not bad

grave jolt
#

well, DateTime consists of Date and Time -- that makes sense to me

paper echo
#

the only bad ones are datetime and logger, off the top of my head. and the builtin types but i guess those are type/function hybrid things anyway

verbal escarp
paper echo
#

that's not how programming languages work

flat gazelle
#

yeah, you don't get to make breaking changes in a programming language

#

elm does it and look at how mad it makes everyone

verbal escarp
flat gazelle
#

python 3 is not an example to follow

grave jolt
#

Unless you're E... yes, you beat me to it

paper echo
#

it's a constant battle against accumulating cruft and garbage, the fact that any programming languages can ever improve at all is a miracle

flat gazelle
#

java 9 is still hard to use due to being even slightly breaking

verbal escarp
#

you know, there's a recipe that works!

flat gazelle
#

java 9... The current java version is java 17

grave jolt
#

ez, just write the language the right way the first time 🤷 🤷

verbal escarp
#
  1. make an alias to have both versions work 2) deprecate the old version 3) remove the old version after 5 years of raising warnings until everyone is annoyed enough to be glad when 4) finally the old version is removed
paper echo
#

that's what python does now, pretty much

unkempt rock
paper echo
#

actually it's a shorter deprecation cycle (3 years i think?) and there was a discussion recently about extending it

flat gazelle
#

yeah, but if you have to wait 5 years between minor changes, that's like 6 versions in 30 years, no wonder old mistakes stick around

#

and like, if you go around breaking everyones code every 5 years, people will stop using the language

paper echo
#

also ironing out the kinks is kind of what python 3 did from 3.0 -> 3.5 or so

#

yes, that

unkempt rock
#

didn't they re-capitalize SOME stuff during that?

paper echo
#

i quipped once that they should have left map and filter as returning lists, and added imap and ifilter that were lazy

#

one of the core maintainers responded to my post and didn't get the joke

#

i was also only half joking

unkempt rock
#

I'd like that so much for casual code

paper echo
#

same

verbal escarp
#

i bet the answer was "WTH ARE YOU TALKING ABOUT JUST USE COMPREHENSIONS IF YOU WANT LISTS"

paper echo
#

no, it was "map() is lazy as of python 3" or something like that

astral gazelle
#

Not rename but a kwarg would be nice

flat gazelle
#

god that would be a mypy nightmare

paper echo
#

@unkempt rock i've put this into a big utils.py before for a couple of projects

def lmap(*args):
    return list(map(*args))

def dmap(*args):
    return dict(map(*args))
flat gazelle
#

dependent typing

paper echo
#

overload + Literal is already a thing 😛

#

dependent python would be funny

verbal escarp
#

did you mean "singledispatch"

flat gazelle
#

still can't typecheck map(lazy=random)

paper echo
#

actually idris 2 has a python compiler target now

astral gazelle
#

Who needs types when i got eager maps

flat gazelle
#

fair enough

#

but honestly, [*map(f, arr)] has served me well, even if it is an abomination

paper echo
#

wait why not just list(map(f, arr))

flat gazelle
#

faster to type

paper echo
#

lol

astral gazelle
#

Unpacking is way cooler

verbal escarp
paper echo
#

i do {**a, **b} and [*a, *b]

#

(i also still want more of the set api for dicts but this has been discussed before and people don't agree with me)

verbal escarp
#

but sadly i'm not people

rich cradle
paper echo
peak spoke
#

I was wondering recently how much of an issue renaming datetime could be

flat gazelle
peak spoke
#

I get not renaming methods as that can quietly break things, but an invalid name should be apparent on a version change

flat gazelle
#

and every one requires a massive codebase change, due to way elm is designed

paper echo
paper echo
rich cradle
paper echo
#

i can't think of any language that has had 19 so-big-the-whole-thing-breaks breaking changes, at least not past the 1.0 mark

flat gazelle
#

well, elm is strictly speaking still in 0. versions

paper echo
#

ah

flat gazelle
#

but it is advertised as being prod ready

#

so I don't think it really gets to hide behind being not a 1.0 yet

paper echo
#

yeah that is an odd way to do things, i'm surprised companies adopt it in that state

grave jolt
#

I think edward kmett has a rant about elm somewhere

paper echo
#

i'm told the elm ui framework is really nice, at least in principle

flat gazelle
#

it is arguably useful for some frontends

#

it more or less removes frontend bugs

#

if you have something in elm, it is likely to be correct for all cases.

grave jolt
#

I think we're going wildly off-topic

paper echo
#

tldr python is better than elm

grave jolt
#

i-

#

ok

paper echo
#

here's a question: is it possible to disable stdin buffering as much as possible, only relying on the os to buffer? -u only seems to disable output buffering

#

use case: cpython is beating the pants off luajit, awk, and common lisp at a task that i am surprised it is doing very well on. and i want to know if it's because it's just slurping huge amounts of input at a time and buffering it (basically cheating on the benchmark i put together), or if it's because it's actually better-optimized than people give it credit for

peak spoke
#

I'm not sure if it'd even be included in python 4 if that'll be a thing

raven ridge
#

What you see as stagnation, I see as stability and usefulness.

#

People who own hundred or thousands of scripts really appreciate not needing to update every one every year.

verbal escarp
#

i don't see it as stagnation but rather lazyness. there's a clear and proven path to modernize things

raven ridge
#

Which is?

flat gazelle
#

when you add more work to your users, they will not like you, is the thing

raven ridge
#

I'm expecting to be done dealing with the Python 2 to 3 migration in 1 to 2 years. I'm still annoyed about that.

verbal escarp
raven ridge
verbal escarp
#

tell that to my code, there are deprecation warnings all the time

flat gazelle
#

just because the work has a longer deadline doesn't mean it's not extra work

raven ridge
verbal escarp
raven ridge
#

No, that doesn't work for code that needs to run in multiple versions.

verbal escarp
#

possibly within under a minute if you use a script..

raven ridge
#

!d DeprecationWarning

fallen slateBOT
#
DeprecationWarning 

exception DeprecationWarning```
Base class for warnings about deprecated features when those warnings are intended for other Python developers.

Ignored by the default warning filters, except in the `__main__` module ([**PEP 565**](https://www.python.org/dev/peps/pep-0565)). Enabling the [Python Development Mode](https://docs.python.org/3/library/devmode.html#devmode) shows this warning.

The deprecation policy is described in [**PEP 387**](https://www.python.org/dev/peps/pep-0387).
visual shadow
#

you just solved legacy code. its strange how it still exists.

raven ridge
#

"ignored by default".

unkempt rock
#

!voiceverify

raven ridge
visual shadow
# fallen slate

im guessing packages dont necessarily use this one, since i could have sworn i've seen depreciation warnings from libraries before, but i dont think it was part of my own code.

verbal escarp
visual shadow
#

that might have been it. i think it was tensorflow in my case

verbal escarp
#

yup

peak spoke
#

Yes breaking old code is an issue, but renaming a class is really one of the least impactful things there is and it's not like it'd be done out of the blue

raven ridge
#

There's a reason that the default behavior is ignoring DeprecationWarning, though. Users don't want to be warned that a library that they're using is doing something it shouldn't be. There are tradeoffs involved.

peak spoke
#

With how python's development is going now, these issues will persist for a long time

raven ridge
#

this is a good thing.

visual shadow
peak spoke
#

If you're concerned about breaking you can remain on the version the script was developed for for a fairly long time

raven ridge
#

4 years.

#

that's not very long, as far as corporations are concerned.

#

I have to work with code that was written multiple decades ago.

visual shadow
#

the balance of managing backward compat is a big responsibility, it's absolutely a tradeoff. people writing code on the edge dont really get to feel the pain of code that's literally written before this millennium for example.

frosty saddle
#

hey guys can someone help me load up a file.

#

or teach how I can

visual shadow
verbal escarp
visual shadow
raven ridge
#

Such things are really limited, in practice. We have experience with that, in the form of 2to3.py - it mostly worked.

#

it also made a ton of unnecessary changes, because it didn't realize they were unnecessary.

verbal escarp
#

how does 2to3 work actually?

visual shadow
#

i've never been brave enough to try those tools to be honest, it just felt so awkward giving up control

verbal escarp
visual shadow
#

test suites would have definitely helped..but ha, what are those... sigh

raven ridge
#

For the people advocating that Python should rename the datetime class: what's the oldest codebase you've maintained? Code written in the 70s, 80s, 90s, 00s, 10s?

verbal escarp
#

for me it's ~20 years, so 10s

#

eh

#

00s?

raven ridge
#

yeah.

#

was that Python code?

verbal escarp
#

C#

peak spoke
#

how big of an issue would a rename(s) like that be? You mentioned the python 2 migration, but the scales of those are not comparable

raven ridge
#

Depends how many of them there are.

#

the 2 to 3 migration included a whole lot of little changes, and a few big ones.

#

little changes like breaking httplib into the separate submodules of the http package.

#

but even the relatively small ones - like removing the long type - have reasonably far reaching impact.

spice pecan
#

The maintenance costs should definitely be considered, but I see no harm in adding an alias. That's far from being confusing and unnecessary, considering that we do have things like collections.namedtuple vs typing.NamedTuple

raven ridge
#

so - you want the name DateTime for the class. Do you also want to rename int, float, str, list, and dict?

peak spoke
#

builtins follow a different naming convention

raven ridge
#

should they?

main ginkgo
#

what if they kept the old name around as a legacy alias with a deprecation warning, giving people a couple versions to add their own datetime = DateTime change to patch it up? though on the other hand i do remember the async change to a hard keyword in 3.7 created a similar kind of mess, especially with d.py

ah nevermind, just saw this was already discussed

peak spoke
#

considering most are used and regarded as functions, yes

raven ridge
#

object is a type implemented in C, datetime is a type implemented in C - what's the difference between them in your mind?

verbal escarp
#

they "feel" like functions most of the time

raven ridge
#

does object?

feral cedar
#

the only common use for it is sentinels

spice pecan
raven ridge
#

so is the complaint the duplication, or is the complaint the fact that it doesn't follow PEP 8 naming conventions?

verbal escarp
#

both ^^

raven ridge
#

ok, let's focus on PEP 8. Should we rename object?

spice pecan
#

For me personally it's the first, I'm fine with built-ins not being pascal case

#

(Though camelcase in logging is a different story)

verbal escarp
#

although, maybe we need to add a chapter about types to PEP8

#

from that angle it would make sense to keep those generics lower case

raven ridge
spice pecan
#

If we were to rename them, I would prefer to have them pascal cased for consistency

unkempt rock
#

consistency
what would you name map?

#

If we're pascal-casing types then that's gonna be Map

verbal escarp
#

if it feels like a function..

feral cedar
#

the issue is how "functiony" something feels

unkempt rock
#

consistent feeling. got it. x)

raven ridge
#

none of these things are "functiony", honestly. int is a constructor that returns an int object.

#

str is a constructor that returns a str object.

verbal escarp
#

you could also regard it as a factory

main ginkgo
#

for me the only real annoyance is when a class/function has the exact same name as the module. socket.socket for example has consistently been a bother.

feral cedar
#

they're listed as built in functions though

raven ridge
#

a constructor is a factory that returns instances of its type.

spice pecan
#

My view on this is the following:
Most if not all built-in types fall either into the category of types that are used as functions (in which case, naming them in lowercase would fit the idea that API clarity is more important than technical details), or types that are integral to the language, which have an all-lowercase name or alias in most languages, so it makes sense to keep them this way. datetime, time and so forth really don't feel like either to me

verbal escarp
raven ridge
#

it's pretty handwavy about "function" versus "type" in that section.

raven ridge
spice pecan
#

And every constructor is a function as well

raven ridge
verbal escarp
raven ridge
#

object isn't a generic type. I'm not sure why you're bringing up generic types.

gleaming rover
raven ridge
#

range isn't, either. Nor int, float, or complex

grave jolt
#

I think this naming convention is sometimes really confusing. For example, lots of people think that range is a generator function for some reason.

gleaming rover
#

(assuming magic backwards compatibility patch)

unkempt rock
#

soo. downcase everything?

gleaming rover
#

realistically speaking

#

there is no way it's going to happen

raven ridge
#

what if something starts as a function, and later becomes a class - do we need to rename it and break all existing users?

spice pecan
peak spoke
raven ridge
grave jolt
#

like, the ones you write

#

or library authors write

raven ridge
#

yes, I am.

main ginkgo
#

b[0, 1, 2] for bytearray literal? anyone? 😛

flat gazelle
#

ambiguous

grave jolt
#

so you're saying we should name classes in snake_case?

feral cedar
raven ridge
spice pecan
grave jolt
#

I'm not asking about this whole issue in general, just about a function morphing into a class

main ginkgo
raven ridge
grave jolt
raven ridge
#

except that now you have two names for one thing. I don't think that's an improvement.

verbal escarp
#

naming is one of the hardest problems in programming 🙂

raven ridge
spice pecan
#

Performance characteristics are indeed very different, but I can't say they pose as types integral to the language - you could have an alternative implementation that would use int lists, the functionality isn't integral in the same sense ints and floats are integral. They are definitely less used, and, just like set, I wouldn't exactly be opposed to having them pascal cased

grave jolt
spice pecan
peak spoke
#

and mutability I guess

verbal escarp
raven ridge
grave jolt
raven ridge
#

because it's a breaking change that affected essentially no one. All previous operations work fine on the new type.

spice pecan
raven ridge
#

Why force them to change?

grave jolt
#

all previous operations
Any mutable operations won't work, as well as isinstance checks

#

but yeah, I guess it's very often not a breaking change

raven ridge
#

well, very nearly all.

#

2to3 replaces every call to range() with a call to list(range()) - but it's almost always unnecessary.

#

In fact, I've never found a case where that transformation was necessary, though I concede that they can exist.

verbal escarp
grave jolt
#

However, in this case changing a function to a class is not the important change. The function will actually return an object that doesn't support operations that the previous one did.

raven ridge
#

Last I heard, pypy is considering supporting Python 2 indefinitely. To give an idea of how much trouble the 2 to 3 migration caused.

red solar
#

fr?

raven ridge
#

I don't know that my info is up to date, but that was the case around a year ago.

red solar
#

it existing isn't support

grave jolt
# grave jolt However, in this case changing a function to a class is not the important chang...

For example, if you had a partial implementation and contract like this: py def partial(fn, /, *args, **kwargs) -> Callable: # the function promises to return a callable! # not necessarily a function def _fn(*args2, **kwargs2): return fn(*args, *args2, **kwargs, **kwargs2) return _fn and then you realized that it's not picklable, you could do this ```py
def partial(fn, /, *args, **kwargs) -> Callable:
return _Partial(fn, args, kwargs)

I guess it's fair to say that every implementation detail will be relied upon by someone, but that's a separate story
raven ridge
#

that's Hyrum's Law.

grave jolt
#

yes

spice pecan
#

I think that if learning something is actually a type and not a function is a surprise to a lot of people, then being a type is likely not a very important detail to know, and the API clarity of being named like a function would possibly be better than conforming to it being a type

grave jolt
#

anyway... I badly need to go to bed

verbal escarp
#

nn

grave jolt
#

we can continue the flame war tomorrow

spice pecan
#

same here

raven ridge
#

is anyone who knows about types surprised that int is a type and not a function? When you're taught types, int is one of the first ones you learn, and you learn to print(type(5)), and it prints that it's an int...

grave jolt
#

not with range and map though IME

spice pecan
#

int isn't the surprising one, range, partial, map, etc are

red solar
#

ah wait before you all leave, i need a quick opinion thing

grave jolt
#

although practically, it may be not that important

spice pecan
#

int is one you'd pretty much always* expect to be exactly int, coming from another language

*Terms and conditions may apply

red solar
# 
buf = bytearray(8)
with atomicuintview(buf) as aiv:
    ...

buf = bytearray(8)
with atomicview(buf, AtomicUintView) as aiv:
    ...

any preference for either style? (hopefully before fix goes to bed)

verbal escarp
red solar
#

AtomicUintView is the type of aiv - there's another 4 type options

#

whereas the top means i have 5 separate functions

spice pecan
#

Why not with AtomicUintView(buf) as aiv?

verbal escarp
#

then i'd say make the second argument into a kwarg and have a default

red solar
red solar
verbal escarp
#

whichever is most common 😄

raven ridge
#

why not take an array.array style format code? 😄

red solar
#

elaborate?

spice pecan
#

!d array.array

fallen slateBOT
#
array.array 

class array.array(typecode[, initializer])```
A new array whose items are restricted by *typecode*, and initialized from the optional *initializer* value, which must be a list, a [bytes-like object](https://docs.python.org/3/glossary.html#term-bytes-like-object), or iterable over elements of the appropriate type.

If given a list or string, the initializer is passed to the new array’s [`fromlist()`](https://docs.python.org/3/library/array.html#array.array.fromlist "array.array.fromlist"), [`frombytes()`](https://docs.python.org/3/library/array.html#array.array.frombytes "array.array.frombytes"), or [`fromunicode()`](https://docs.python.org/3/library/array.html#array.array.fromunicode "array.array.fromunicode") method (see below) to add initial items to the array. Otherwise, the iterable initializer is passed to the [`extend()`](https://docs.python.org/3/library/array.html#array.array.extend "array.array.extend") method.

Raises an [auditing event](https://docs.python.org/3/library/sys.html#auditing) `array.__new__` with arguments `typecode`, `initializer`.
red solar
#

(i've never used array)

spice pecan
#

:P

#

The idea is that there's a string code that specifies the format

red solar
#

.fromlist()? like that?

#

oh

raven ridge
#

it takes a type code as a required argument, and has type codes for signed short, unsigned short, signed int, unsigned int, unsigned char, etc

main ginkgo
#

what about a good old memoryview

red solar
#

so more like atomicview(buf, "uint")?

raven ridge
#

there's also the struct module with format codes.

spice pecan
spice pecan
grave jolt
#

ah yes, the regex of serialization

verbal escarp
#

strings as modes are brittle

red solar
#

enum would be ideal, hmmm, maybe i should do that

raven ridge
#

realistically, the best answer for your users, given that you have a constrained set of types, is probably a separate type for each at the top level. So, my vote would be for AtomicUnsignedIntView

grave jolt
#

yeah, you can type-hint enum members with Literal

verbal escarp
#

no help from IDEs on typos etc

red solar
grave jolt
#

but alas

red solar
raven ridge
#

No, I'm advocating for not having atomicview at all, or at least not exposing it to users.

red solar
surreal sun
#

What is this atomicview thing btw? I’ve been hearing it here a lot

raven ridge
#

users would only see with AtomicIntView(buf) as aiv:

red solar
surreal sun
red solar
verbal escarp
spice pecan
raven ridge
tulip inlet
#

Hey sorry to bother you but i have a little probleme on Sony Vegas can someone help me pls?

surreal sun
verbal escarp
raven ridge
#

realistically, the entire bring-your-own-buffer approach is very, very weird. Especially given that the buffer needs to be aligned, and such. You probably shouldn't expose any such API to Python users in the first place.

spice pecan
red solar
# raven ridge in what way is it not friendly to IDEs? 
aiv = atomicview(buf, UINT)
aiv.store(5)

@raven ridge in your version this isn't prevented by the types, it's prevented by checking internally if __enter__ has been called and then raising an exception
whereas if aiv is a type that only supports __enter__ and __exit__, it's prevented by what little type system we have

red solar
verbal escarp
surreal sun
red solar
#

sure

tulip inlet
#

in wich that's the question

red solar
raven ridge
#

Well, then, why not just AtomicIntView.__enter__() -> EnteredAtomicView, where EnteredAtomicView has store(int)?

spice pecan
#

The question for me is, why not still have Atomic<Type>View as the main type, and then have Atomic<Type>View.__enter__ -> Atomic<Type>ContextView (Exactly what geek said, yeah)

grave jolt
#

I think a thread would be really good here ngl

red solar
#

idk how that works

spice pecan
#

Can we make those here?

red solar
#

(unless that was a pun?)

grave jolt
spice pecan
#

Sad

grave jolt
#

that was just a thought

red solar
raven ridge
#

What doesn't work about this for type hinting?

red solar
# 
def inc(a: AtomicUintView):
    a.inc()

with atomicview(buf, UINT) as aiv:
    # type(aiv) -> AtomicUintView
    inc(aiv)
#

idk seems nicer to me than AtomicUintViewContext in inc

#

(realistically that would be more like Union[AtomicIntegral, AtomicIntegralView] or something)

raven ridge
#

Well, 🤷 - I'm just going to go back to saying that I think the entire bring-your-own-buffer approach is awkward in Python, and not a good idea. And if you didn't have that, you wouldn't have any of these problems.

red solar
#

does python have a way to import enum members in a way such that you can use them without the underlying class?

red solar
#

but no way to like make enum members globals?

surreal sun
red solar
#

damn :/

verbal escarp
#

uhhhm..

spice pecan
#

Hmm... You could have something along the lines of a prepare classmethod, so that

AtomicTypeView.prepare(buffer) -> AtomicTypeViewContext
AtomicTypeViewContext.__enter__ -> AtomicTypeView

with AtomicTypeView.prepare(buffer) as atv:
    ...
raven ridge
verbal escarp
#

actually, you can assign global variables to your Enum member

red solar
red solar
verbal escarp
#

and then you can import those while still having them as members

spice pecan
#

It is, why wouldn't it be?

red solar
#

mostly just thinking of C's enums

red solar
verbal escarp
#

i put enum members as class attributes in a different class, no problem

surreal sun
raven ridge
#

a C enum member isn't a first class object at all, it's just an integral value of a specific type

red solar
#

if i do MEMBER = TheEnum.MEMBER and then use MEMBER

red solar
verbal escarp
# 
>>> from enum import Enum
>>> Foo = Enum("Foo", "a b c")   
>>> a = Foo.a
>>> a
<Foo.a: 1>
spice pecan
#

Linters should infer the type from that, yeah

red solar
#

ok lemme try that, cuz honestly i'm happy with atomicview(buf, UINT)

#

with the .prepare() thing being close second

#

also @raven ridge if you don't want to provide a buffer, a = AtomicInt(width=4) is already available and you can pass it around as a normal variable 🙂

raven ridge
#

with the bring-your-own-buffer approach - is there a way to get the child process and parent to share a buffer on Windows? And, how will you share it on Unix - is it fork? If so, couldn't you just as easily fork an AtomicInt object that itself holds the buffer?

red solar
#

can't really have an inbetween

raven ridge
#

why would you ever want anything other than "the user has no access to the buffer"?

#

if the user looks in the buffer, it's undefined behavior, right?

red solar
#

yes

raven ridge
#

(other than through your APIs, that is)

red solar
#

but say i have a program using shared memory in C++, and then I want to also interact with that shared memory in Python, the user would need to provide a buffer of that shared memory, there's no other way

#

this isn't just for shared memory between python processes

raven ridge
#

"no" other way? What about a constructor that takes a shmkey?

red solar
#

shmkey?

raven ridge
#

the argument you provide to shmat

red solar
#

ah, i've only ever used shared memory with boost::interprocess

#

i don't have a reason for not being able to do that... but that seems to lead to me needing to have a constructor that supports every platforms specific way of accessing shmem?

#

or of supporting both mmap at least? and then maybe shared_memory constructors?

raven ridge
#

I don't know Windows at all, I'm a Unix guy. But for everything-but-Windows, system v shared memory segments have a unique integer ID assigned to them, and things can attach to them using the ID.

verbal escarp
raven ridge
red solar
#

idk, i don't have a great answer, really it's just it was a lot of effort and seemed too specific, whereas telling the user to provide their buffer and just keeping it active for the duration of the context seemed a good balance

red solar
red solar
verbal escarp
#

your own

red solar
#

ok

verbal escarp
#

but akin to numpy

raven ridge
#

and if that's the case, there's no BYOBuffer use case for MAP_ANONYMOUS

red solar
#

i mean, ok i'll look through the mmap module, one sec

raven ridge
#

well, anyway: yes, my suggestion is to have a way to construct your AtomicInt with each supported platform's shared memory identifier. If your AtomicInt is in charge of doing the "attaching" and getting the pointer, it makes resource management easy, because your destructor will do the detaching.

red solar
#

i think it's a lot of unnecessary effort to support all the platforms instead of byob - i don't think it's worth supporting all the potential ways of constructing it, however i can yield on a special case of passing an AtomicInt to another process within python...

#

so either have AtomicInt allocate from shared memory, or have AtomicSharedInt... (probably the former)

#

but i still wanna keep byob

raven ridge
#

I think expecting Python users to byob is just gonna get you segfaults. It's a very difficult interface to use.

red solar
#

one sec

raven ridge
#

and one that's outside their wheel house.

red solar
# 
ctx = atomicview(buf, dtype.uint)
# if ctx.release() is called, ctx.__enter__() will raise an exception
with ctx as a:
    # ctx.release() will raise an exception here
    ...
print("done")
# a.load() will raise an exception (as will any usage of a)
#

the only way to get a segfault is if you break the contract of "keep the buffer around for the duration of the with context"

#

and realistically you can't even get rid of the buffer once you create ctx, CPython will raise an exception

raven ridge
#

Python prevents the buffer from being garbage collected once you create the ctx

#

but they allocated it in some way, and they can still free it in the same way.

#

munmap or shmdt or whatever.

red solar
#

buf has to support the buffer protocol

#

CPython prevents any modifications to the buffer (like resizing or releasing it) while someone else has a "reference" to it

raven ridge
#

I mean, no

#

the implementer is required to do that - but there's nothing that stops the implementer from getting it wrong

#

it's not "prevented"

#

it's "forbidden"

red solar
# 
>>> buf = bytearray(8)
>>> m = memoryview(buf)
>>> buf += b'a'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
BufferError: Existing exports of data: object cannot be re-sized
raven ridge
#

yes, because the bytearray type gets it right.

#

but whatever type you're expecting the user to bring might not.

#

that's not Python enforcing that, it's bytearray enforcing it.

red solar
#

i mean, if someone gets to the point where they're using their own special made type that supports the buffer protocol, and then they mess up, i kinda feel that's on them

raven ridge
#

what else would they ever do?

red solar
#

use shared_memory or mmap?

raven ridge
#

there's no use case where it would ever make sense for them to use bytearray, is there?

red solar
#

both of which support memoryview (and the buffer protocol in general)

#

are you sure this isn't an implementation thing tho? CPython doesn't keep track of Py_GetBuffer and PyBuffer_Release calls?

raven ridge
#

what would it do based on that?

#

the bytearray needs to know that there's been more Py_GetBuffer calls on it than PyBuffer_Release calls, so that it can reject calls to __iadd__

red solar
#

hmm ok ig nothing

#

yeah

#

idk, i think you can screw up if you really try even by just providing shmkey or whatever it was, and this is fairly far fetched, so i think it's safe enough

#

interesting tho, always thought the BufferError was on the interpreter's end

zinc shale
#

Hi is anybody here alright with multiprocessing and shared memory in python?

red solar
#

lol

verbal escarp
zinc shale
#

xd

red solar
#

i mean, seeing the past 30m of discussion...

zinc shale
#

Like i have a problem with accessing an array

raven ridge
#

@red solar for that it's worth, also - I don't agree with those who said that your original approach was unfriendly for type checkers. open() constructs a file object, the file object has a __exit__ that calls close(), and if you call a method after close() or after the context is exited, you get exceptions. Your type wouldn't be any weirder than the file objects returned by open

red solar
#

and having both the IDE warn them and an exception, rather than just an exception is ideal 🙂

red solar
zinc shale
#

like i can't accesse the array within temp

#

thats a bare code i have like 400 lines between them lines xd

red solar
#

try putting global arr1 on the first line of the function?

#

oh wait the temp is run in a new process?

zinc shale
#

yes

#

so u can't do that

red solar
#

you pass temp to the new process, but you don't copy arr1 over... so there's no way for the new process to access it i think

#

you'll need to pass arr1 as an argument over to the new process

zinc shale
#

hmmm

fallen slateBOT
#

Hey @zinc shale!

Uh-oh! It looks like your message got zapped by our spam filter. We currently don't allow .txt attachments, so here are some tips to help you travel safely:

• If you attempted to send a message longer than 2000 characters, try shortening your message to fit within the character limit or use a pasting service (see below)

• If you tried to show someone your code, you can use codeblocks
(run !code-blocks in #bot-commands for more information) or use a pasting service like:

https://paste.pythondiscord.com

main ginkgo
main ginkgo
red solar
paper echo
#

if i have calc.py and calc.cpython-310-darwin.so in the same directory, which one does python import?

paper echo
#

apparently it's .so for anyone interested

pliant tusk
#

What if there is a pycache for calc.py?

paper echo
#

good question, unsure

#

i think it still goes for the compiled version

pliant tusk
#

Makes sense

#

But still interesting

paper echo
#

when it comes to "do file i/o and a small amount of arithmetic", cpython is pretty fast. this is 3m data points

#

for some reason graal was really slow, maybe it doesn't buffer stdin as aggressively

#

(i assume cpython and other implementations buffer stdin aggressively, i don't really have a way to verify)

red solar
verbal escarp
#

i was just wondering, considering https://github.com/beartype/beartype#pep-484-deprecations (which can't really be made backwards-compatible with aliases since Optional, Union etc. comes with new syntax) - would/could/should it work that justuse only runs on python >=3.10 but if someone wants to use() a package that requires an older python version that a sub-interpreter is spawned for that package-venv?

GitHub
GitHub - beartype/beartype: Unbearably fast O(1) runtime type-check...

Unbearably fast O(1) runtime type-checking in pure Python. - GitHub - beartype/beartype: Unbearably fast O(1) runtime type-checking in pure Python.

red solar
#

I don’t have an answer, but is typing.List actually being deprecated?

grave jolt
#

...Importing those from typing is deprecated.

red solar
#

Why tf tho… there’s no issue in just having them

grave jolt
#

having two ways to do something is something Python seems to move on from

#

like, deprecating @asyncio.coroutine

#

(for now let's just ignore that we have 4 ways to format strings)

red solar
#

Honestly how tf do people support multiple Python versions when features get deprecated left right and centre

grave jolt
#

messy ifs

#

and, well, those things from typing will only be removed in 2025 or so

#

so we have plenty of time to adapt

verbal escarp
# red solar Honestly how tf do people support multiple Python versions when features get dep...

exactly my question about how we should deal with that in https://github.com/amogorkon/justuse - would it make sense to use sub-interpreters to bridge the gap?

GitHub
GitHub - amogorkon/justuse: Just use() code from anywhere - a funct...

Just use() code from anywhere - a functional import alternative with advanced features like inline version checks, autoreload, module globals injection before import and more. - GitHub - amogorkon/...

red solar
#

But I don’t see how you would like communicate between two interpreters with different Python versions?

grave jolt
naive saddle
verbal escarp
# grave jolt so we have plenty of time to adapt

that's the problem - it's not a future thing but it's right here, right now. the temptation is great to drop python <3.10 right away, but most scientific tooling is still on 3.8 or lower (looking at conda)

naive saddle
#

.. or we just reimplement functions that are too new

grave jolt
#

babel for python when

red solar
verbal escarp
#

no clue.

red solar
#

(And are you limited to going back to the first version subinterpreters were introduced?)

#

Oh

verbal escarp
#

@lusty scroll might be deeper in the theory

#

i'm thinking of a subprocess of some sort and then communicate via stdin or somesuch

#

there's the subinterpreter PEP but i somewhat doubt it's for this kind of scenario

red solar
#

So either only support packages compatible with the current version of Python, or throw performance out the window?