internals-and-peps

[unkempt rock]

?

[haughty shuttle]

Excuse me for disturbance

But I need help

pleaseeeeeee

@unkempt rock

[sacred yew]

a: dont ping some random person
b: actually describe your question instead of begging for help
c: this is the wrong channel, get a help channel via #❓｜how-to-get-help

[verbal escarp]

btw, is there a recommended pastebin for python these days? on irc it was bpaste all the way

now i'm looking for a pastebin extension on vscode, but it's all a bit meh

especially for long pytest outputs it'd be nice

[split chasm]

I personally use upaste.me usually

Hastebin is pretty popular

Probably better

!paste this discord also has one

[fallen slate]

Pasting large amounts of code

If your code is too long to fit in a codeblock in discord, you can paste your code here:
https://paste.pydis.com/

After pasting your code, save it by clicking the floppy disk icon in the top right, or by typing ctrl + S. After doing that, the URL should change. Copy the URL and post it here so others can see it.

[verbal escarp]

ty

[surreal sun]

dis documentation is quite useful for understanding bytecode

so the reason why

def outer():
  a = 0
  def inner():
    a += 1

fails regarding bytecode is that __iadd__ is returning a + 1, and considering that you're redefining a, it presumes it's in the local, but a doesn't exist in the local and it calls LOAD_NAME leading to an UnboundLocalError

[pliant tusk]

in order for that to work, you need to specify to python that a isnt a local (using nonlocal a)

[surreal sun]

yep

but behind the bytecode, is that why?

[pliant tusk]

yes

!e py import dis dis.dis(''' def outer(): a = 0 def inner(): a += 1''') dis.dis(''' def outer(): a = 0 def inner(): nonlocal a a += 1''')

[elder blade]

Yes too, because a += 1 is the same as a = a + 1 although it makes the assumption that a has to be local.

[fallen slate]

@pliant tusk :white_check_mark: Your eval job has completed with return code 0.

001 |   2           0 LOAD_CONST               0 (<code object outer at 0x7f5fe39cfa80, file "<dis>", line 2>)
002 |               2 LOAD_CONST               1 ('outer')
003 |               4 MAKE_FUNCTION            0
004 |               6 STORE_NAME               0 (outer)
005 |               8 LOAD_CONST               2 (None)
006 |              10 RETURN_VALUE
007 | 
008 | Disassembly of <code object outer at 0x7f5fe39cfa80, file "<dis>", line 2>:
009 |   3           0 LOAD_CONST               1 (0)
010 |               2 STORE_FAST               0 (a)
011 | 
... (truncated - too many lines)

Full output: https://paste.pythondiscord.com/kozuyevoqi.txt?noredirect

[sand goblet]

It’s weird how it does that with lists too even though it ends up just mutating them, not making another one

[pliant tusk]

its for consistency

[languid yoke]

It doesn't do type inference when it's generating the bytecode.

[surreal sun]

!e

from dis import dis

dis("""
list=[]
list.append(2)
""")

[fallen slate]

@surreal sun :white_check_mark: Your eval job has completed with return code 0.

001 |   2           0 BUILD_LIST               0
002 |               2 STORE_NAME               0 (list)
003 | 
004 |   3           4 LOAD_NAME                0 (list)
005 |               6 LOAD_METHOD              1 (append)
006 |               8 LOAD_CONST               0 (2)
007 |              10 CALL_METHOD              1
008 |              12 POP_TOP
009 |              14 LOAD_CONST               1 (None)
010 |              16 RETURN_VALUE

[surreal sun]

it looks like it builds a list during the bytecode

[pliant tusk]

[] generates a BUILD_LIST opcode

if python did type inference then it would replace LOAD_NAME ... CALL_METHOD opcodes with LIST_APPEND

[surreal sun]

ah

[unkempt rock]

i have a question

someone sent me this file

[fallen slate]

Hey @unkempt rock!

It looks like you tried to attach file type(s) that we do not allow (.bat). We currently allow the following file types: .gif, .jpg, .jpeg, .mov, .mp4, .mpg, .png, .mp3, .wav, .ogg, .webm, .webp, .flac, .m4a.

Feel free to ask in #community-meta if you think this is a mistake.

[unkempt rock]

can someone check it out

its a .bat

(dms)

or i can copy the stuff

[sharp flume]

this will try to delete all content on your drives except the C drive

[astral gazelle]

why would you post this here or anywhere

[verbal escarp]

is there a good way to combine icontract with typeguard or something similar?

i like icontract a lot, but isinstance checks shouldn't be necessary with proper annotations..

[main lynx]

agree they shouldn't be necessary, but static type checking can cover most cases, depending on what you're doing

checking validity of data structures does not seem like something you should be doing in icontract anyway, those checks are bypassed in production but you probably still want to validate the data in production

[verbal escarp]

then what are you suggesting instead?

[main lynx]

i don't have a suggestion because i've never wanted for runtime type checking

pydantic gets good reviews for transforming untrusted data into typed Python classes

and validating

so you lean on mypy to static check your business logic, pydantic to validate input types, and icontract to validate the values in your logic and the data

then again, if you're using icontract to validate values in the data, this also goes away in production

pydantic has custom validators, so in theory you should use those for data and icontract only for the business logic

[verbal escarp]

i mostly want to validate my algorithms during development and have meaningful test output

without polluting my code with asserts

[main lynx]

i guess i don't understand why running mypy on it isn't sufficient

[verbal escarp]

mypy isn't very ergonomic

[main lynx]

how so?

[verbal escarp]

we gave up on mypy after we ended up with ugly code that was full of basically meaningless annotations only to make mypy happy

we tried typeguard, which worked quite well, but it seems incompatible with icontract

[main lynx]

i did have some trouble adapting code written without mypy to fully annotated and checked, but only when abusing a particular "clever" generic generator interface

i think i ended up # type: ignore a few spots and moving on

[candid pelican]

heyo..

i've a question, there's some way to write a class with methods will be inherited like an abstractmethod but isn't mandatory these methods exists but they cannot be different of the super class's methods ??

[boreal umbra]

So, you're talking about methods that a subclass is not required to have, but if the subclass does have it, it has to be the exact one from the parent class (no subclass-specific implementation)?

[candid pelican]

ye exactly

[raven ridge]

https://docs.python.org/3/library/typing.html#typing.final does that for static type checkers, but it isn't enforced at runtime.

[boreal umbra]

You don't want to have subclasses that deliberately exclude functionality from the parent class.

[raven ridge]

right, that breaks the Liskov Substitutability Principle.

[candid pelican]

yessir

[boreal umbra]

But if you sure you want it anyway, you would need to implement the method to raise an exception

[candid pelican]

i wanna make a class with http methods

[boreal umbra]

class Foo:

    def dont_include_in_subclass(self):
        # do stuff idk. but the subclass shouldn't have this method


class Blah(Foo):

    def dont_include_in_subclass(self):
        raise NotImplementedError

[candid pelican]

like this:

class HTTP:
  def get():
    pass

  def post():
    pass

class Foo(HTTP):
  def baz():
    pass

it should raise an exception

bc the method baz do not exist in HTTP class

how can i do that?

[boreal umbra]

Python's design discourages you from preventing people from doing stuff if they want to do it. So, the short answer is, "don't make a baz method if you don't think there should be one".

[raven ridge]

It's technically possible to do that (with __init_subclass__), but you really shouldn't. Making a class that has weird restrictions placed on what it's allowed to contain will confuse people who try to use your HTTP class.

[candid pelican]

im asking bc a saw a library... theres the methods and you cannot type methods that do not exists in the inherited class

[sand goblet]

Does this need to have a post on bugs.python.org? Or is it minor enough to be considered trivial? There’s an X next to the pull request saying it should have a post on bugs.python.org and have a blurb about it, unless it’s trivial

[surreal sun]

Or, the name mangling version that causes other problems if you don't want something that doesn't go into any subclass

class Foo:
  def __dont_include_in_subclass(self):
    # do stuff

[boreal umbra]

Does the name of methods get mangled as well? And of course it doesn't work if the method is intended to be part of the api

[surreal sun]

wdym?

[boreal umbra]

Double underscored names fall under the same convention as single underscored names, in that they should be treated as non public

[surreal sun]

double underscored names are usually for avoiding inheritance collission:

[boreal umbra]

Yes, and they're not treated as public. What if you want it to be treated as public?

[surreal sun]

!e ```py
class Foo:
def __bar():
pass

def _baz():
pass

def baz():
pass

class Bar(Foo):
pass

print(Bar.dict)
b = Bar()
print(b.dict)

huh

[fallen slate]

@surreal sun :white_check_mark: Your eval job has completed with return code 0.

001 | {'__module__': '__main__', '__doc__': None}
002 | {}

[surreal sun]

oh well

[surreal sun]

I mean you technically could access it outside

self._Foo__bar

but

your way is better

much better probably

especially for a package

[boreal umbra]

Yes, I'm strategically avoiding the word "private"

[surreal sun]

mb

[elder blade]

Pydantic?

[verbal escarp]

there's a far better way to do this - by delegation/composition

you could use a variant of @singledispatch on your methods to delegate to certain functions on certain conditions or you could pass functions to be used as methods on your class construction (or "install" them after construction) or you could modularize everything with enums and dispatch based on state.. endless possibilities

so there's no "excluding of functionality" per se but rather a "nothing to see here" approach

[wise locust]

hi

[verbal escarp]

hi

[fleet ice]

Is checking if the input is in the grid better than try except?

[wise locust]

hi

[verbal escarp]

what grid?

it's just a minimal example of public vs. "private" vs. mangled

[fleet ice]

Matrix

@verbal escarp

[verbal escarp]

uh hu

[safe stag]

I have a kinda general question I've been thinking about. I don't think it fits a help channel so I thought I'll try here.
I have a python codebase in a single git repo. Some of it is a django app that calls out to other modules. Some of it is just plain Python. I try to make the django part as "layer-y" as possible - just an interface to my other code.
How do you structure such codebases? I mean, literally in the filesystem.

[surreal sun]

It was a test

[fleet ice]

Is checking if the input is in the grid (2 by 2 matrix) better than try except?

[verbal escarp]

could you provide an example of what you mean?

[leaden willow]

life is a simulation

[spiral palm]

True!

[fleet ice]

board = [[None for _ in range(3)] for _ in range(3)]

def move1(x, y):
  try:
    print(board[y][x])
  except:
    exit()

def move2(x, y):
  if x > 3 or x < 0 or y > 3 or y < 0:
    exit()
  print(board[y][x])```

[verbal escarp]

why didn't you ask that in a help channel? 😉

[boreal umbra]

@fleet ice take a look at #❓｜how-to-get-help

Unless you're just asking for your general knowledge of how exception handling compares to if statements. In which case my suggestion would be to try using %timeit in IPython. I'd actually be interested to see the result.

[fleet ice]

I meant by better is "suggested" way of doing things

[fiery stirrup]

Hello everyone, I have seen frameworks like Chaquoty that allow to run python in AndroidStudio, but are they valid alternatives for a real production environment in Android app?

[candid pelican]

quick question... dataclass vs BaseModel?

[mint forge]

whats BaseModel?

[charred pilot]

pydantic

[sand goblet]

If you want to add a comment to python source code, that’s considered a trivial change, right?

[paper echo]

Attrs

I wouldn't use dataclass over attrs

The question is whether you want validation-on-instantiation and nice class syntax, or not

[verbal escarp]

hey @paper echo 🙂

we're almost finished cleaning up the code in justuse, we got 95% test coverage, i'd say 0.5 is due next week or so

all tests pass, too

which means i'm fairly confident that things won't come crashing down

i'm running step-by-step debugging sessions to find any remaining inconsistencies from the many refactorings we did

[paper echo]

That's great to hear

Testing something like that isn't trivial

[verbal escarp]

parametrization ftw 🙂

it also helps to have a guy like @lusty scroll, well-versed in git and multi-platform testing, taking care of those things. alone i probably wouldn't have gotten to that point in 4 years, realistically

synergy \o/

[sand goblet]

https://dev.azure.com/Python/cpython/_build/results?buildId=87934&view=results
Is the failure because of this? https://bugs.python.org/issue41682
Or is it not?

[rich cradle]

Whoever just deleted their message, !e needs to be at the beginning of the message

[candid pelican]

what is the algorithm used in the sort method??

foo = [4, 0, 2, 1]
foo.sort()
print(foo) # [0, 1, 2, 4]

[native flame]

timsort

[candid pelican]

thanks

[sullen bronze]

What does repl it uses? How it renders browser in website?

[timber spindle]

not sure if right channel, but deos anyone have experience with cryptography?

[timber spindle]

basically im trying to decode this link

9 83. 57 85 / 23 12 145 77 1 q

P'T AOL TVZA MHTVBZ VM TF RPUK, P MLHY AOHA DOPJO PZ ULCLY YLZAPUN ULCLY ZAPSS TVCPUN ZPSLUASF MYVT OPSS AV OPSS, PA KVLZ UVA DHSR, YBU VY AYVA HUK SPCL PU H WSHJL AOHA KPZHWWLHYZ PU AOL KHYR HUK HWWLHYZ PU AOL KHF. FVB JHUUVA MLLS PA PM FVB AVBJO PA

before that was this, which i decoded to get

IM THE MOST FAMOUS OF MY KIND I FEAR THAT WHICH IS NEVER RESTING NEVER STILL MOVING SILENTLY FROM HILL TO HILL IT DOES NOT WALK RUN OR TROT AND LIVE IN A PLACE THAT DISAPPEARS IN THE DARK AND APPEARS IN THE DAY YOU CANNOT FEELIT IF YOU TOUCH IT

[timber spindle]

gottaa decode this rn loll

its part of the problem

that was given to me to decode

yup, i figured that out, but the 7 letters

doesnt apply to the lnk, i guess

instructions:

yup its mod 26 i belive, it wraps around

yeah which would be 7 the other way

[unkempt rock]

This isn't the right channel, this channel is for a discussion about Python itself

[uncut sage]

When dicts were changed to preserve insertion order, did the same thing happen with sets? My experiments seem to indicate not. If so, why/why not?

[visual shadow]

Same thing did not happen with sets.

For reasons I'd have to read up on it, I've forgotten now.

If I recall correctly, I think dicts being insertion ordered was actually just a side effect, it wasn't why the implementation was changed

And then the decision was whether to deliberately make it shuffle items to make it seem like dicts still aren't insertion ordered even though the new more efficient implementation was. This was decided against

[uncut sage]

Gotcha. Just checking that I haven't missed something

[visual shadow]

https://stackoverflow.com/questions/61414947/why-dont-python-sets-preserve-insertion-order this is a great read.

[paper echo]

i do wish there were more ordered and frozen collections in the stdlib

(i still use OrderedDict for ordered mappings even in new versions of python)

[peak spoke]

Do you use it for the explicitness of relying on the order or do you sometimes needs the operations on both ends it provides?

[paper echo]

the former

but i guess also the latter if i needed it. i very rarely need it

[peak spoke]

I saw some del dict[next(iter(dict))] recently in I think the lru cache implementation

[paper echo]

fwiw most of the time i end up with Iterable[Tuple[KeyT, ValueT]] and don't need the mapping functionality

[lusty scroll]

@verbal escarp @paper echo

[verbal escarp]

xD

[paper echo]

...what am i looking at

is this justuse installing itself?

[peak spoke]

I've also found that I end up iterating and passing items around with dicts more often that not lately

[verbal escarp]

yup xD

[paper echo]

lol

[paper echo]

wdym by that?

[peak spoke]

After creating it a lot of operations end up being a comprehension over the dict items for filtering or something similar to process it for the user and then the filtered item tuples are passed along for e.g. a display func

[paper echo]

ah, yep

maybe that's the same as what i do

generators w/ key,value tuples, collect into a dict at the end

[verbal escarp]

i'd like to throw @assert ... in the room as an idea

[pliant tusk]

so @assert ... instead of assert ...?

[verbal escarp]

https://icontract.readthedocs.io/en/latest/introduction.html basically does asserts before and after calling a function, they hope that design-by-contract to be part of the language at some point, so i was thinking that maybe one could have assert basically as a decorator for this purpose

they link https://groups.google.com/g/python-ideas/c/JtMgpSyODTU

what i gather, it's mostly about adding to annotations though

[paper echo]

*ahem*
#internals-and-peps message
#internals-and-peps message

@signature
def atan2(y: float, x: float) -> float:
    """This is where the docstring goes."""

@signature
def atan2(y, x) -> Assertion[
    not math.isclose(x, 0) or not math.isclose(y, 0),
    "X and Y both cannot be 0."
]: ...

@signature
def atan2(y, x) -> Assertion[
    (
        0 <= __return__ <= math.pi
        if y >= 0
        else -math.pi < __return__ < 0
    ),
    "y in [0, pi] if nonnegative, else in (-pi, 0) -- the overall range is (-pi, pi]."
]: ...

def atan2(y, x):
    # Implementation
    ...

i don't think this needs to be part of the core language - plenty of libraries do it well enough

python-ideas is kind of a cesspit of people pitching "special interest" proposals

[main lynx]

most languages do not include DBC features afaik

and it would not seem to advance Python's goal of being a rapid prototyping language

otoh people who have invested time into Python are interested in it being taken seriously for all kinds of production uses

i include myself in that

Python post-conditions seem kind of complicated, you'd need to be able to make assertions about generator yields and return values separately

signature looks cool but i think part of the purpose of DBC features or libs is consuming less LoC than just writing assert statements, and that's a lot of LoC for basically two assert statements and a regular annotation/docstring

post-condition decoration has an advantage over assert that it checks all possible places you may have returned from

like all verification tools it requires complete line coverage in tests to confirm all branches

and at that point why didn't you just write a unit test for post-conditions

DBC going the extra mile of generating tests to cover all branches would have value

[paper echo]

i agree it's a lot of LoC

part of the issue with python is its syntactic rigidity for things like this

lack of tidy lambda syntax really bites

[paper echo]

that's a good point too - you'd probably have to have it auto-detect the kind of function you're annotating, since generator return values are almost useless (or you could write __yield__ instead of __return__)

tha said, i think something this actually plays to the strengths of python

you can add the contracts incrementally as you "solidify" your app

afaict that's how it works in any DBC framework anyway

i could probably write Precondition and Postcondition instead of Assertion to make the distinction clearer

yet another reason to use hy instead of python 😛 you could easily wrap this up into a nice lispy API

[main lynx]

six characters of lambda keyword does seem like a python quirk relative to other popular langs

but tbh i think it's really good

imagine you're a python noob

you see the lambda keyword, you google it, you get answers

now imagine you're a javascript noob

you see (x, y) => { ... }

what do you google for?

[paper echo]

Good point

=> probably comes up with stuff but yeah

The reason I proposed using the type annotation is because it's one of the only places in python that captures unevaluated expressions

Basically it avoids the need to write lambda

[surreal sun]

why is it called lambda? lambda is a name for some linea ralgebra term iirc

could've called it anon imo

[main lynx]

intresting, so you can have Validator[this is Python code run with your set of locals/globals]

[paper echo]

It's from lambda calculus, theoretical computer science stuff

[surreal sun]

hm

[flat gazelle]

lambda is the conventional name for this coming from lisp

[paper echo]

I'm pretty sure? I think it's just saved as a string literal

There might be some limitations on it, it might still have to be syntactically valid

But I know for a fact that name lookup doesn't happen anymore

That's how you can have unquoted forward references

[main lynx]

if it needs to be syntactically valid that suggests it gets "compiled" at import

[paper echo]

Right, but it's not executed

[trail ridge]

Is this bad:

foo = self.foo
bar = self.bar
foo_bar = self.foo_bar

# some logic that changes values for foo, bar, and foo_bar

self.foo = foo
self.bar = bar
self.foo_bar = foo_bar

My variable names are long. I guess I should also try to make them shorter.

[paper echo]

no, i do that all the time. but i try to keep "mutating" behavior isolated from other behavior and obvious from the API

def Thing:
    def update_from_db(self, db_conn):
       ...

a name like update suggests that it's self-modifying

[trail ridge]

When you do the above, do you do that so it's easier to read?

Yeah, I will also add update in the method name. Thanks!! That would tell us from the get go that the method will update and mutate properties!

[surreal sun]

just curious, did you mean class Thing?

[paper echo]

yes, hah

[surreal sun]

Hm, so if I"m understanding correctly

the block allocation API for CPython essentially has different classes for the size of something, and places them in their pools depending on the size, which are grouped into arenas

and if there isn't a pool available, it creates one

[white nexus]

Hey y'all. If you were choosing a library to use for models, when would you choose one of marshmallow or pydantic over the other?

I've been looking at marshmallow to use, since it has many additional configuration options, and a huge plugin library of sqlalchemy and environment integrations

However, pydantic uses typehints, which is excellent.

[grave jolt]

This is actually better than mutating attributes in some way -- it will never place the object into an inconsistent state even if there's an exception in the middle

[surreal sun]

How would you create a random number generator with just plain code and logic, not using anything external

Genuine question, I'm curious as to the logic behind them

Ik tjhe random module uses marvelle twister? I forget the name lol

[paper echo]

yes, the "mersenne twister" algorithm is kind of the standard one

[paper echo]

true, this style is kind of "transactional" in a way

[paper echo]

marshmallow is only for "runtime" serialization/deserialization. pydantic is basically attrs/dataclass + marshmallow at the same time

@white nexus you could use attrs+desert or attrs+cattrs or dataclass+mashumaro or dataclass+dataclass-factory to emulate what pydantic does

there's a lot of appeal in pydantic being "all in one"

but maybe it's overkill for times when you don't need serialization/deserialization and/or strong validation, and you just want less boilerplate when creating classes

iirc pydantic has fewer options for control of serialization/deserialization compared to marshmallow too

you also might want to consider integration (or lack thereof) with jsonschema

it's kind of an open and active space right now

[white nexus]

so why use attrs+desert over marshmallow?

I was also leaning towards marshmallow due to marshmallow sqlalchemy integration

[paper echo]

desert generates a marshmallow schema from an attrs class

!pypi desert

[fallen slate]

desert v2020.11.18

Deserialize to objects while staying DRY

[surreal sun]

turns out random.randrange under just uses os.urandom or something then shifts the bits

[paper echo]

that's one reason i like desert - it just glues together existing tools

[white nexus]

but why not

!pypi marshmallow-dataclass

[fallen slate]

marshmallow-dataclass v8.5.3

Python library to convert dataclasses into marshmallow schemas.

[paper echo]

never knew about it 🤷‍♂️

i know the creator of desert from freenode

i don't think he ever moved to libera

so i'm biased towards it

[white nexus]

i mean, that is linked in desert

[paper echo]

oh let me see

oh i know

[paper echo]

because marshmallow-dataclass doesn't or didn't support attrs

[white nexus]

So I guess, why use attrs over dataclass?

[surreal sun]

I never understood "pickling" and "deserialization" and "serialization", what does that mean? does that just mean converting from a non readable format to a datatype

[paper echo]

more features, faster, supports __slots__ by default

[white nexus]

I am probably building a way overkill configuration system lmfao

[paper echo]

serialization is turning "python stuff" into "generic data", deserialization is the opposite. so you might "serialize" an object to JSON

yes, pickling is serialization to a custom binary format

[surreal sun]

ah

and __getstate__ and __setstate__ are used for pickling?

[paper echo]

that i don't know

[surreal sun]

!d object.getstate

[fallen slate]

object.\_\_getstate\_\_

object.__getstate__()```
Classes can further influence how their instances are pickled; if the class defines the method [`__getstate__()`](https://docs.python.org/3.10/library/pickle.html#object.__getstate__ "object.__getstate__"), it is called and the returned object is pickled as the contents for the instance, instead of the contents of the instance’s dictionary. If the [`__getstate__()`](https://docs.python.org/3.10/library/pickle.html#object.__getstate__ "object.__getstate__") method is absent, the instance’s [`__dict__`](https://docs.python.org/3.10/library/stdtypes.html#object.__dict__ "object.__dict__") is pickled as usual.

[surreal sun]

Ah

[paper echo]

oh, that's useful

weird names but useful

[peak spoke]

The only place where urandom should be involved if you're not using SystemRandom is seeding

[surreal sun]

Ah

[white nexus]

@paper echo so uh, out of curosity, how do you define a nested marshmallow field on a attr class when using desert?

[white nexus]

okay i figured that out

[white nexus]

hmmmmmm

for some reason, I'm having a lot of trouble with validation

as in, using desert and attrs or dataclasses just does not care if the variable is required or not

wait

i had the wrong method

[exotic cypress]

Hi all, not sure if this is the best channel for general advice or strategy on a project that I am thinking of undertaking.

The end result will be a map, segmenting different local government areas that shows a couple of different buckets of backyard sizes in each area.

I have a GeoJSON dataset from Bing that is building footprints for about 11m+ properties in Australia. I'll also look to get property outlay datasets from the Aust. Govt so I can calculate for each residential property the actual area, minus the building shape from Bing.

(If I could get Google GeoJSON of Building Footprints I would).

In terms of questions or advice sought:

• Additional Resources that you might think will be helpful.
• I don't need all 11m+ datapoints - only those that reside in residential areas (otherwise I'll be importing 6GB+ before conducting operations) . Would it be best for testing and the like to limit this to a particular state and figure out how to just import the GeoJSON datapoints that relate to that state?
• How should I store calculations locally after calculating? In another GeoJSON file?
• Is GeoPANDAS the best framework for looking to calculate and manipulate this data?
• Recommendations for other communities to discuss this with.

[paper echo]

@exotic cypress maybe #databases or #software-architecture . But PostGIS is a thing

[exotic cypress]

Thank you!

In terms of databases that I have worked with in the past - it is 100% limited to SQL Relational. I've never touched anything outside of Microsoft SQL (being a student, not a lot of experience).

If I'm reading the cliff notes on PostGIS correct, it's built on top of PostgreSQL which is a RDS, and PostGIS is an extension on top of that.

Considering that this will be for implementation and execution on my local machine (for a uni assignment) - do you think I'm going to have any performance issues? Or because of query folding and the like this is going to be shit hot better than just reading/writing GeoJSON?

[paper echo]

no, definitely don't bother with a database if you don't need one

[white nexus]

@paper echo desert and marshmallow-dataclasses were causing some bugs with marshmallow features...
The most important one affected for my usage was partial.

Passing true to partial did not work with dataclasses and desert as it does without them

[white nexus]

I just had an idea

What if I don't use dataclasses

And just type my schemas

[paper thorn]

Hi, I am stuck on this problem where I am building a package out of a module using setuptools. Can anyone help?

[boreal umbra]

@paper thorn take a look at #❓｜how-to-get-help

[hexed spire]

<@&831776746206265384>

That's a phishing link ^^

He sent those in several channels

#community-meta

Or am I wrong and it's not a phishing link?

Weell eivl said it's not a phishing link sorry

[lusty scroll]

Is there a supported way to create/manipulate pathlib Path on *nix? This comes up empty:

>>> p = pathlib.Path("D:\\")
>>> p
PosixPath('D:\\')
>>> p.drive
''

[peak spoke]

You can use a PureWindowsPath/WindowsPath directly

[lusty scroll]

thanks, I didn't realize that was part of the API

[old hearth]

what's a good introduction to "client-server" program architecture with practical examples ?

[loud zenith]

Hey

Could someone navigate me to where the code gets executed in CPython

I looked under Parser/ but i can only see the tokens and tokenizers
I cant see the actual execution of commands

Or does Parser/ contain the main interpreter file, 🤔

I looked under Python/ But that seems to be the standard library

[surreal sun]

It depends on where you want it from

for code getting executed

compile.c compiles the code to bytecode iirc

[white nexus]

There is no good way to do this

[loud zenith]

Oh great

So it was under Python/

Hm

so my main interpreter should be in the same folder as the standard library?

Well i guess ill just do that then

So like

Parser/
Pix/
    main.py
Mac/
Windows/
...

Wait a minute

Nvm

Ill just follow that

[true ridge]

Basically the flow is

Parser/tokenizer.c -> Parser/parser.c -> Python/compile.c -> Python/ceval.c

[true ridge]

Not for CPython, only the core modules (sys/builtins) reside in the same directory (which is probably an historical artifact) as the evaluation loop (core of the interpreter).

[viscid matrix]

Hi Pythons !
I have a question for Flask users:
What is the common way to structure the database procedures logics (insert, delete, edit, etc):

1. in the respected model, and use the route method as a controller to manage the model procedures (because there might be more than one sometimes).
2. Do everything in the router method.

[white nexus]

SO IT TURNS OUT.
I needed to define a default to the variables.

of = marshmallow.missing

[surreal sun]

How would a class with __str__ differentiate when it is being represented (I.e print(class_instance)) versus being converted to a string str(class_instance)

[peak spoke]

It wouldn't, they use the exact same mechanism for the conversion. The only choice would be inspecting the source or call stack

[surreal sun]

Ah, that sucks.
Why doesn’t __repr__ take more precedence? It’s made for representing the object

[gleaming rover]

isn’t that what repr is for

[surreal sun]

__repr__ is a fallback for __str__

if you have __str__ defined, it'll be the same behavior regardless if it is str(class_instance) or print(class_instance)

[gleaming rover]

no

the free function

!e print(repr(['a']))

[fallen slate]

@gleaming rover :white_check_mark: Your eval job has completed with return code 0.

['a']

[peak spoke]

str would usually be what you'd want the user you're printing for to see, defaulting to repr would only be good for developers which in most cases should have logging set up anyway

[surreal sun]

i suppose

Oh wait that gives me an idea for a decorator, something like singledispatch except for the context of when it's called

def run_in_print():
    return 1

def run_in_for():
    return range(1, 20)

@contextdispatch({"print": run_in_print, "for": run_in_for})
def foo():
    return 2

i got the basics working, where it'll run:
run_in_print if foo() is printed,
run_in_for if foo() is used in a for loop,
and if none of these use cases apply, it uses the basic foo() itself

[elder blade]

That's not its only purpose?

There's also __format__ right?

[surreal sun]

__format__ is for class_instance.format though right?

[elder blade]

No, it's when you use '{0}'.format(custom_class)

[surreal sun]

Ahh

My bad

[elder blade]

It's what allows you the '{0!r}'.format(custom_class) IIRC

Let me find the documentation for it hold on

https://docs.python.org/3/reference/datamodel.html#object.__format__

This applies to __format__ https://docs.python.org/3/whatsnew/2.6.html#pep-3101-advanced-string-formatting

[surreal sun]

hm, interesting

[paper echo]

repr, str, and pformat are all on a spectrum from "ugly but has technical info" to "pretty and human readable"

You could use repr to emit python code that can be copied and pasted to re-create an object, while you might use str to emit some prettier representation

[white nexus]

this is probably cursed but

!e ```py
oof = 'oof'

print(type(oof))
#<class 'str'>

print(str(type(oof)))

"<class 'str'>"

How can i get exactly `str` from the type of `'oof'`

[fallen slate]

@white nexus :white_check_mark: Your eval job has completed with return code 0.

001 | <class 'str'>
002 | <class 'str'>

[peak spoke]

I'm not sure I understand but def.__name__ is a thing

In [24]: str.__name__
Out[24]: 'str'

[surreal sun]

TIL you can set an attribute to a function (useful for decorators within a function being a wrapper)

def foo():
     pass

foo.hi = "a"
print(foo.hi)
>>> "a"

[elder blade]

That's what's called first-class functions

[surreal sun]

IIRC functools.singledispatch uses attribute setting on a function

[peak spoke]

It wasn't always possible as it's an extension type

[surreal sun]

It was added since docstrings had to be used for other information and Python core devs didn’t like that, so they added attribute setting, right?

[white nexus]

rather than <class 'str'> I want to get str

[peak spoke]

what is str

[white nexus]

!e ```py
print(str.name)
print(type('hi'))
print(type('hi').name)

wtf

[fallen slate]

@white nexus :white_check_mark: Your eval job has completed with return code 0.

001 | str
002 | str

[white nexus]

👀

[surreal sun]

str.name is a string lol

[white nexus]

oh

lmao

I had an extra str() in my source

[boreal umbra]

I'm increasingly of the impression that the terms "high level" and "low level" languages doesn't tell you nearly as much as "imperative" vs "declarative".

[spark magnet]

most adjectives used to describe programming language are used as if they were black and white, but they are all subtle and squishy

[boreal umbra]

Feel free to continue, if there's more to that.

[spark magnet]

"object-oriented" for example: it's not yes or no

[surreal sun]

What does it mean by “imperative” and “declarative” for programming langauges

[spark magnet]

"functional", "declarative", "strongly typed", etc.

[boreal umbra]

right. if encaps defines OOP for someone, then Python isn't really that.

[spark magnet]

python has fine encapsulation

(see?)

[boreal umbra]

but does it have
d a t a h i d i n g?

[spark magnet]

not as much as other languages. but some.

[boreal umbra]

"the code describes instructions for the computer" vs "the code describes properties of the result of some calculation"

[surreal sun]

Ahhh

Like breaking something done

Down

[boreal umbra]

like, mean(some_list) is declarative. You want the average of the values in the list. You didn't stipulate that a 0 needs to be allocated somewhere, and that each value in the list needs to be added to that running sum, etc.

Am I wrong?

[spark magnet]

if you showed python of today to an assembly language programmer from 1950, would they think it was declarative?

[white nexus]

thank you all for the assistance, my configuration system is finally starting to work.

[boreal umbra]

it's more declarative than assembly, but idk what words mean what in 1950.

[white nexus]

In retrospect I should have looked at a tool that uses configuration from multiple sources and files.....

like coverage, pytest, flake8.....

[spark magnet]

idk if the way coverage did it is a good way 🙂

[white nexus]

ah well, no better time to reinvent the wheel than now except its an oval not a circle

[boreal umbra]

(it's funny because nedbat maintains coverage.py, but maybe you know this.)

[white nexus]

I do, I wouldn't have realised that other tools do it if I had not seen him in chat 😛

[spark magnet]

🙂

[white nexus]

lol, I was making a system which takes input from both .env files and a config.toml file. The problem was lying in my validation, and also having a default configuration that I wanted to partially validate. I ended up with a lot of compromises, and a huge mess

But its my mess

[boreal umbra]

just gotta hope there's a cross section of your oval that's a circle and try to use that

[spark magnet]

i know the feeling 🙂

[white nexus]

def _build_class(klass, class_prefix: str = "") -> Bot:
    # get the attributes of the provided class
    attribs: typing.Set[attr.Attribute] = set()
    for a in dir(klass.__attrs_attrs__):
        if hasattr(klass.__attrs_attrs__, a):
            if isinstance(attribute := getattr(klass.__attrs_attrs__, a), attr.Attribute):
                attribs.add(attribute)
    # read the env vars from the above
    with env.prefixed(ENV_PREFIX):
        kw = defaultdict(lambda: marshmallow.missing)  # any missing required vars provide as missing
        for var in attribs:
            kw[var.name] = getattr(env, var.type.__name__)(class_prefix + var.name.upper())

        return klass(**kw)

this.... mess.... takes an input attr.s class and creates it from environment variables if they exist. If they don't exist, it sets the variables to marshmallow.missing so marshmallow knows they aren't there.

I only spent roughly 13 hours figuring out that my dataclasses needed to have a default value of marshmallow.missing if they didn't have any other defaults.

[white nexus]

with what I wrote, coverage.py probably does it better....

using atoml, attrs, desert, environs, and marshmallow for this simple thing 😔

ah well, hopefully no one will need to maintain this in the future

[white nexus]

WTF

I managed to get a KeyError on a default dict.

[boreal umbra]

(said in a stern voice) what did you do?

[white nexus]

was trying to make a defaultdict default to defaultdicts

so I used a lambda

defaultdict(lambda: defaultdict))

oops

[boreal umbra]

I'm surprised that even worked

[white nexus]

this is what I have now

def _generate_default_dict():
    """For defaultdicts to default to a defaultdict."""
    return defaultdict(_generate_default_dict)

[boreal umbra]

Oh god

[surreal sun]

Would anyone mind giving feedback to something I created?

[surreal sun]

O_o

[white nexus]

...its not that bad is it?

[boreal umbra]

Only after I banish @white nexus to #esoteric-python

[white nexus]

smh

is that really esoteric?

[surreal sun]

So basically, I created something like functools.singledispatch but for context with how the function was called (uses regex)

For example:

@contextdispatch
def bye(parameter):
    pass

@bye.register(r"nah\(\d+\)")
def hi(parameter):
    return "hi"

print(bye(12))

This will call hi(12) as when bye(12) is called, it will check if the context of where it was called fits the regex specified under register, so nah(digit that is greater than a length of 1), and when I called it, it is satisfied, hence it calls hi instead of bye

[boreal umbra]

I'm tired and I don't understand it. So yes.

[white nexus]

I needed a defaultdict of defaultdicts

[surreal sun]

this is just an example, I know it isn't useful lol with this example, but i had a few examples where it was quite useful

[white nexus]

so I made a function to return a new default dict and uses itself as the factory

so any time its needed, it returns a new defaultdict referencing what generates them

[surreal sun]

I would make a PEP for this cause I think it could be useful but idek how to lmao

And I feel like they won't accept PEPs from someone who isn't even old enough to graduate from secondary school

Maybe I could go to the python-ideas mailing list and talk about it, but I feel like I"m going to get roasted lol

Hello!

[paper echo]

It's not about age, I wouldn't worry about that. But code that acts differently depending on its syntactic context is... weird

Interesting, fun, but weird

[surreal sun]

True

[paper echo]

Probably not something you want in the core language

[surreal sun]

Ah that's true lol

I might just put it on PYpi incase anyone wants to use it

I made it mostly because I wanted to differentiate my class instance from calling __str__ with str(class_instance) versus __repr__ for representing the class instance itself

and then I expanded it

I could see it being useful in some cases though, hence i might put it on PyPI

Probably the coolest thing I've made recently though

[paper echo]

Why not use repr() when you want repr?

I'm impressed that you could make such a thing

[surreal sun]

Because __str__ will be the default, meaning that by default, I can't differentiate between str(class_instance) and print(class_instance), it calls __str__ for both

Since __str__ takes precedence over __repr__

I'm misinterpreting what you say I think

[white nexus]

print(repr(class_instance))

[surreal sun]

That would be good for internal usage, except that if my user doesn't specify that, they would just get __str__ which isn't what I would want

[paper echo]

i thought this was for your own code

this would be madness and chaos in "real" code

[surreal sun]

I certainly wouldn't see it being used in prod anywhere but sometimes I might forget to do repr(class_instance)

[surreal sun]

Yeah, all you have to do I think is get from the stack as it'll push the function call

Full source code:

from re import search


def contextdispatch(function):
    contexts = {}

    def register(context: str):
        def register_decorator(func):
            contexts[context] = func

            def running_register(*args, **kwargs):
                return func(*args, **kwargs)

            return running_register

        return register_decorator

    def run_function(*args, **kwargs):
        function_called = stack()[-1].code_context[-1].rstrip('\n')
        for context, func in contexts.items():
            if re.search(pattern=fr"{context}", string=function_called):
                return func(*args, **kwargs)
        else:
            return function(*args, **kwargs)

    run_function.register = register
    return run_function

I'm kinda proud of it but I don't see it being used that much lol

@contextdispatch
def foo():
    return "hello"

@foo.register(r"print\(foo\(\)\)")
def run_in_print():
    return 1

@foo.register(r"for . in foo\(\)")
def run_in_for():
    return range(1, 20)

print(foo())
a = foo()
print(a)
for i in foo():
    print(i)

oh god I just realized the potential for screwing up code this has lol

I could probably create some kind of object with this with just functions

[white nexus]

#internals-and-peps message

that itself hurts me to see lol

[paper echo]

that won't recurse infinitely until explosion?

!e ```python
from collections import defaultdict

def _generate_default_dict():
"""For defaultdicts to default to a defaultdict."""
return defaultdict(_generate_default_dict)

d = _generate_default_dict()
print(d['a'])

[fallen slate]

@paper echo :white_check_mark: Your eval job has completed with return code 0.

defaultdict(<function _generate_default_dict at 0x7f46824d8040>, {})

[paper echo]

interesting

[white nexus]

nope, its a factory

[paper echo]

i'm too old for this shit

[white nexus]

!e ```py
from collections import defaultdict

def _generate_default_dict():
"""For defaultdicts to default to a defaultdict."""
return defaultdict(_generate_default_dict)

d = _generate_default_dict()
print(d['a']['a']['a']['a']['a']['a']['a']['a']['a']['a']['a'])

[fallen slate]

@white nexus :white_check_mark: Your eval job has completed with return code 0.

defaultdict(<function _generate_default_dict at 0x7fd7ba861040>, {})

[paper echo]

i see

that's pretty clever

[white nexus]

i initially used a lambda

but that did not work since it was not recursive

i have done terrible things today....

[paper echo]

!e ```python
from collections import defaultdict
from functools import partial
d = defaultdict(partial(defaultdict, defaultdict))
print(d['a']['a']['a']['a']['a']['a']['a']['a']['a']['a']['a'])

[fallen slate]

@paper echo :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 4, in <module>
003 | KeyError: 'a'

[paper echo]

heck

i see

that's a pretty cool hack actually

nice one

you could use this to implement javascript-style dicts/objects

or maybe not, because it needs to recurse forever

[white nexus]

I used attrs with marshmallow, desert, environs, and atoml to make a configuration system that makes me want to delete system32

[paper echo]

well it's fun anyway

!pypi environs

[fallen slate]

environs v9.3.3

simplified environment variable parsing

[paper echo]

!pypi atoml

[fallen slate]

atoml v1.0.3

Yet another style preserving TOML library

[paper echo]

neat

this seems like the seed of a good configuration system

[white nexus]

environs is python-dotenv but implements marshmallow
atoml is tomlkit but more recently updated

[white nexus]

until you see this

def _build_class(klass, class_prefix: str = "") -> Bot:
    # get the attributes of the provided class
    attribs: typing.Set[attr.Attribute] = set()
    for a in dir(klass.__attrs_attrs__):
        if hasattr(klass.__attrs_attrs__, a):
            if isinstance(attribute := getattr(klass.__attrs_attrs__, a), attr.Attribute):
                attribs.add(attribute)
    # read the env vars from the above
    with env.prefixed(ENV_PREFIX):
        kw = defaultdict(lambda: marshmallow.missing)  # any missing required vars provide as missing
        for var in attribs:
            kw[var.name] = getattr(env, var.type.__name__)(class_prefix + var.name.upper())

        return klass(**kw)

which does something and I already forgot what it does

uhhhhhh

[paper echo]

i looks like you're dynamically building a new class using an existing attrs class in order to apply default arguments that are dynamic based on env vars, but only the env vars that were in place when the app was started and not when the class is actually instantiated (in case they changed)

[white nexus]

i think it takes a attrs decorated class and for all attr.Attributes, checks if there's a env var--- yeah

wait no

[white nexus]

*making an instance of klass from the env vars

[surreal sun]

What is defaultdict

!d collections.defaultdict

[fallen slate]

collections.defaultdict

class collections.defaultdict(default_factory=None, /[, ...])```
Return a new dictionary-like object. [`defaultdict`](https://docs.python.org/3.10/library/collections.html#collections.defaultdict "collections.defaultdict") is a subclass of the built-in [`dict`](https://docs.python.org/3.10/library/stdtypes.html#dict "dict") class. It overrides one method and adds one writable instance variable. The remaining functionality is the same as for the [`dict`](https://docs.python.org/3.10/library/stdtypes.html#dict "dict") class and is not documented here.

The first argument provides the initial value for the [`default_factory`](https://docs.python.org/3.10/library/collections.html#collections.defaultdict.default_factory "collections.defaultdict.default_factory") attribute; it defaults to `None`. All remaining arguments are treated the same as if they were passed to the [`dict`](https://docs.python.org/3.10/library/stdtypes.html#dict "dict") constructor, including keyword arguments.

[`defaultdict`](https://docs.python.org/3.10/library/collections.html#collections.defaultdict "collections.defaultdict") objects support the following method in addition to the standard [`dict`](https://docs.python.org/3.10/library/stdtypes.html#dict "dict") operations:

[white nexus]

def _build_bot_class(klass: typing.Any, class_prefix: str = "", defaults: typing.Dict = None) -> Bot:
    """
    Create an instance of the provided klass from env vars prefixed with ENV_PREFIX and class_prefix.

    If defaults is provided, uses a value from there if the environment variable is not set or is None.
    """

[paper echo]

_config_original_defaults = {
    'thing1': 'a',
    'thing2': 'b',
}

_config_env_defaults = {}

def _get_config_default(key):
    return _config_env_defaults.get(key, _config_original_defaults[key])

def _populate_config_defaults_from_env():
    # populate _config_env_defaults
    # idk how this env thing works
    ...

@attr.s()
class Config:
    thing1 = attr.ib(default_factory=lambda: _get_config_default('thing1'))
    thing2 = attr.ib(default_factory=lambda: _get_config_default('thing2'))

if __name__ == '__main__':
    _populate_config_defaults_from_env()
    ...

i imagine you could do it like that too, right?

[white nexus]

probably

I mean, I load the toml file into a dict-- and that's why I needed to make defaultdicts if the key did not exist.

that's because I do this to the dict of toml loaded values

unparsed_user_provided_cfg["bot"].update( attr.asdict(_build_bot_class(Bot, "BOT_", unparsed_user_provided_cfg["bot"])) )

and the bot key may not exist if it was not defined in the toml

[white nexus]

probably, similar to that would work, I suppose

[white nexus]

The complexness with this is because it validates the environment variables
getattr(env, var.type.__name__)(class_prefix + var.name.upper())

environs.Env has validation, with Env.bool, Env.string, and so on and so forth for all primitive types

this entire mess is to get the attributes of the class, and more specifically the type of the attribute, so I can getattr on the Env object to succesfully validate the variable

[paper echo]

interesting

there's probably a more elegant way to combine this

[white nexus]

there defintely has to be one lol

[paper echo]

have you considered not trying to bake the env-provided defaults into the attrs class?

[white nexus]

wdym?

There's several configuration classes here, some nested, but only one of them (The Bot config) supports environment variables right now
all of the others will be configured either from the database source or from the local toml, or their defaults.

[paper echo]

1. define Config with only the original defaults in it, none of the env var overrides
2. load the toml data via marshmallow, leave optional things optional in the marshmallow schema
3. fill in defaults provided by env vars before serializing from dict to attrs via marshmallow

that said, there seems to be some overlap here between environs and marshmallow

[white nexus]

Environs depends on marshmallow

[paper echo]

ah, right

[white nexus]

although I do wish there was a better integration to take env vars to a marshmallow schema

[paper echo]

but it doesn't appear to define a schema using marshmallow? it just looks like it uses the validators in the docs

[white nexus]

environs doesn't really do that...

[white nexus]

yeah, pretty much.

[paper echo]

this library is a bit "imperative" rather than "declarative", and i think you could roll this yourself with marshmallows more efficiently

[white nexus]

The reason I have it here is for a few env vars which need to be loaded before the whole configuration system

Specifically I need to load a few developer variables before loading the whole system

and I wanted to make that as little code as possible but meh, could probably roll my own ig

[white nexus]

imperative vs declarative?

[paper echo]

env.int("TTL", validate=lambda n: n > 0)

this looks like it "does something" - it doesn't declare what something should be, it actually takes an action and maybe fails or maybe doesn't

[white nexus]

(The variable(s) in question is the LOG_LEVEL and maybe the disabled/enabled loggers.)

[paper echo]

that or i totally misunderstand the API

are there actually docs or just these examples?

[white nexus]

raises validationError if TTL cannot be converted into an int, or is less than or equal to 0.

[paper echo]

right, but does that immediately read the env var when it's run (imperative-ish and dynamic)? or does it just set up a specification for the env var when eventually the env vars are read (declarative and static)?

imperative: "please do X, then please do Y"
declarative: "X must happen before Y"

[white nexus]

I'm not sure fully what you mean so I'm going to defer you to this: https://github.com/sloria/environs#deferred-validation

By default, a validation error is raised immediately upon calling a parser method for an invalid environment variable. To defer validation and raise an exception with the combined error messages for all invalid variables, pass eager=False to Env. Call env.seal() after all variables have been parsed.

[paper echo]

it still reads the env vars one at a time

as opposed to setting up a specification for all the expected env vars, then reading them all

like what you'd do with a hand-written marshmallow schema

[white nexus]

well, I looked in the source and it loads the .env all at once into os.environs

[paper echo]

well of course, python already does that

[white nexus]

but that's because it just wraps python-dotenv

[paper echo]

oh, the .env file

[white nexus]

ye

[paper echo]

still, it's not the same thing

this is more like

foo = os.environ.get('FOO')
if not foo_is_valid(foo):
    raise Ouch('foo')

bar = os.environ.get('BAR')
if not bar_is_valid(bar):
    raise Ouch('bar')

and less like

class Specification:
    foo: Foo
    bar: Bar

[white nexus]

ah

hmmm

I had this from earlier to get a list of all env vars that were for this program

env = environs.Env(eager=False, expand_vars=True)
env.read_env(".env")
env_vars: typing.Dict[str, str] = {}
for key, value in os.environ.items():

    if key.startswith(ENV_PREFIX):
        env_vars[key[len(ENV_PREFIX):].lower()] = value

the problem I basically was having was splitting an env var since they cannot contain . in their name

well actually they can but its not really supported

[paper echo]

wdym?

were you trying to put . in env var names?

[white nexus]

I did for a little while lol

the problem with the env var conversion is every thing else is dot-delimited

the attr classes use .,
getting a value from the classes is with dots
the toml file uses dots...
etc

to load an environment variable, there has to be special parsing to figure out the _ vs the . ---
should it be a _ or is this actually a .?

I'll probably come back to maybe changing how the environment variables later; right now the code works so I want to get it to work with the existing codebase

@paper echo okay yeah I'll probably almost certainly change it-- I'm not even using the environs validation right now lmao

i have eager off, and never call seal

hmmmm

I need to merge the two better than this...

self.config.user.bot.prefix or self.config.default.bot.prefix

[paper echo]

fwiw toml isn't great for deeply nested data

the environ plugin parses the dots for you?

[white nexus]

meh, its not super nested. And its only for local overrides in the event a user doesn't want to use the database for configuration

this is my basic configuration file right now

[bot]
token = "NjQzOTQ1MjY0ODY4MDk4MDQ5.342b.4inDLBILY69LOLfyi6jk420dpyjoEVsCoModM"
# prefix = "?"

[colours]
base_embed_color = '0xffffff'

[dev.mode]
production = false
develop = false

[paper echo]

yeah, there are lots of ways to do this

generating ad-hoc classes is a creative one!

[white nexus]

(that's a very fake token for anyone wondering)

[white nexus]

one of my requirements for this was to be able to type out all of my configuration

I've met that goal.

[paper echo]

hah, fair enough

[white nexus]

self.config.user.bot.prefix

my final task after making it was to make one god variable

@attr.s(auto_attribs=True, slots=True, kw_only=True)
class Config:
    user: Cfg
    default: Cfg


config = Config(user=USER_PROVIDED_CONFIG, default=DEFAULTS_CONFIG)

this allows me to have the single config variable globally, with the defaults one property away. This also allows me in the future to put a database configuration or something else on the same global config variable

even better, the user class automatically has the defaults part of it

but if its dumped with marshmallow, then it doesn't show all of the default variables

@paper echo i just pushed the change, yeet. https://github.com/discord-modmail/modmail/commit/7016de3a5eef447eb7f4210749a92132d0e6e225

[paper echo]

nothing wrong with having config all in one variable imo

[white nexus]

oh god I just realised the merge conflicts this is gonna cause. No matter which gets merged first the second one needs to implement the new configuration

[loud zenith]

Oh thanks

[loud zenith]

So where is the main file for CPython that executes the commands 🤔
Basically im following this article -> https://ruslanspivak.com/lsbasi-part1/
And im not so sure where to place the Interpreter class

Got it

Thanks!

Thats what im trying

for my own programming langauge

So far so good, i understand what tokens do
And im not copy pasting his code

Im just implementing what he does in my own way

which is why i dont understand what class Foo(object) does different from just class Foo:

[raven ridge]

In Python 3, nothing. They mean exactly the same thing.

[loud zenith]

Oh thanks

[raven ridge]

"old style class" vs "new style class", yeah.

[loud zenith]

yeah that guy seems to be using Py 2 :c

[raven ridge]

But all classes in 3 are new style.

[loud zenith]

Old class?

🤔

Yeah

but since im not really copy pasting his code

its not much of a hurdle

:D

token_names = [
    "INTEGER",
    "PLUS",
    "EOF"
]

class Token:
    def __init__(self, type: str, value) -> None:
        '''
        Type can be any of the token names
        Value can be any char
        '''
        self.type = type
        self.value = value

    
    def __str__(self) -> str:
        # Example: Token(INTEGER, 3)

        return f"Token({self.type}, {self.value})"

See, its a bit different :)

(im using the token_names array because CPython uses an array for the token names too)

const char * const _PyParser_TokenNames[] = {
    "ENDMARKER",
    "NAME",
    "NUMBER",
    "STRING",
    "NEWLINE",
    "INDENT",
    "DEDENT",
    "LPAR",
    ...

Oh yeah whats the dunder repr method

is it the same as __str__

Oh i see

so its good to have the dunder repr anyways

and just return the dunder str

[raven ridge]

!e they're similar, but no.

x = "hello world"
print(x.__str__())
print(x.__repr__())

[fallen slate]

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

001 | hello world
002 | 'hello world'

[loud zenith]

Ah

Does it return the raw string?
so x = "\nhello world"
and x.__repr__()) would return '\nhello world'

?

[raven ridge]

!d object.repr

[fallen slate]

object.\_\_repr\_\_

object.__repr__(self)```
Called by the [`repr()`](https://docs.python.org/3.10/library/functions.html#repr "repr") built-in function to compute the “official” string representation of an object. If at all possible, this should look like a valid Python expression that could be used to recreate an object with the same value (given an appropriate environment). If this is not possible, a string of the form `<...some useful description...>` should be returned. The return value must be a string object. If a class defines [`__repr__()`](https://docs.python.org/3.10/reference/datamodel.html#object.__repr__ "object.__repr__") but not [`__str__()`](https://docs.python.org/3.10/reference/datamodel.html#object.__str__ "object.__str__"), then [`__repr__()`](https://docs.python.org/3.10/reference/datamodel.html#object.__repr__ "object.__repr__") is also used when an “informal” string representation of instances of that class is required.

This is typically used for debugging, so it is important that the representation is information-rich and unambiguous.

[loud zenith]

:O

[raven ridge]

__str__ is "convert this object to a string". __repr__ is "return a string explaining what this object is".

[unkempt rock]

i realized good code , is code that i'm scared to write

[loud zenith]

Ahh so when people want to get the official string representation they do repr(Foo) and not str(Foo) because then Foo would be a string? 🤔

[raven ridge]

In both cases it returns a string, but the meaning of the string is different.

When possible, that's the normal thing to do, yeah.

[paper echo]

repr is meant to be the "ugly but machine-readable-ish" representation of an object. str is meant to be the "human-readable" version. godlygeek did it better

[loud zenith]

Then woudnt doing this

    def __str__(self) -> str:
        # Example: Token(INTEGER, 3)

        return f"Token({self.type}, {self.value})"

    
    # Good to have both methods,
    # Defaults to other if one is not present
    def __repr__(self) -> str:
        return self.__str__()

Be considered bad?

Since its.. returning the same thing?

[paper echo]

no, but godlygeek's explanation is better because they made the semantic different clear

str is conversion, repr is representation

[loud zenith]

Oh i see, but from the article i was following it was returning self.dunder str in dunder repr

    def __str__(self):
        """String representation of the class instance.

        Examples:
            Token(INTEGER, 3)
            Token(PLUS '+')
        """
        return 'Token({type}, {value})'.format(
            type=self.type,
            value=repr(self.value)
        )

    def __repr__(self):
        return self.__str__()

[loud zenith]

Yep

Ahh so when people want to get the official string representation they do repr(Foo) and not str(Foo) because then Foo would be a string? 🤔

oh wait, i just realised repr is short for representation

[raven ridge]

If you have a class Person: that has attributes like first_name and last_name, you might have a __str__ that returns f"{self.first_name} {self.last_name}" and a __repr__ that returns f"Person(first_name={self.first_name}, last_name={self.last_name})"

For instance.

[raven ridge]

Both str and repr return strings.

[loud zenith]

Ah, so in my __repr__ should i return f"Token(type={self.type}, value={self.value})"?

[raven ridge]

That's a reasonable repr, yeah.

[loud zenith]

Then what about str

:c

when would someone wanna use str

[paper echo]

maybe

f"Token(type={self.type!r}, value={self.value!r})"

[loud zenith]

!r 🤔

[paper echo]

another way to put it: repr is for programmers, str is for users

[loud zenith]

Hm

Got it guys

i should keep both separate then

[paper echo]

or define __repr__ and omit __str__

[raven ridge]

They probably wouldn't. Converting that to a str isn't a reasonable operation, other than to get a string representation of it, so it makes sense to not define __str__, and to let it fall through to use __repr__ instead.

[loud zenith]

    def __str__(self) -> str:
        # Example: Token(INTEGER, 3)

        return f"Token({self.type}, {self.value})"

    
    # Good to have both methods,
    # Defaults to other if one is not present
    def __repr__(self) -> str:
        return f"Token(type={self.type!r}, value={self.value!r})"

[raven ridge]

The only reason to define both is if you want them to return two different things.

[loud zenith]

But like, what does !r do

[paper echo]

!r uses repr() instead of str() when doing the string interpolation

[loud zenith]

Ah, so when you try printing it, it calls __str__?

Ok got it

Aight cya

ima head back to making the interpreter :)

[paper echo]

!e ```python
from pathlib import Path
p = Path()
print( repr(p) )
print( str(p) )

[fallen slate]

@paper echo :white_check_mark: Your eval job has completed with return code 0.

001 | PosixPath('.')
002 | .

[south junco]

is there a code that spaces automatically when entering a number in Matrix Elements?

while True:
    row_size = int(input("Enter the row size of the Matrix:"))
    col_size = int(input("Enter the columns size of the Matrix:"))
    matrix = []

    print('\033[31m' + '    put space between each number' + '\x1b[0m')
    print("Enter the Matrix Elements: ")
    for i in range(row_size):
        matrix.append([int(j) for j in input().split()])
    Sum = 0
    for i in range(len(matrix)):
        for j in range(len(matrix[0])):
            if i < j:
                Sum += matrix[i][j]
    print("Sum of Lower Triangular Matrix Elements is: ", Sum)

[visual shadow]

this is not a help channel. please see #❓｜how-to-get-help

[south junco]

sorry I'm new to this

[visual shadow]

all good 🙂

[white nexus]

...

@paper echo my test suite is minimal. Marshmallow has 70+ warnings for my lil configuration system 🥲

really wish I had seen these warnings when I was developing, still not sure how to enable them

oh.

actually they're all caused by desert

guess I.... have a future pr to make.

[novel vector]

How do you use aes in current python? Even pycryptodome docs shows Crypto.Cipher.AES but seems that Crypto is from pycrypto which is deprec.

[surreal sun]

Conventionally, __repr__ is used to show how the object was built, iirc

!e

class Test:
  def __init__(self, foo, bar, baz):
    self.foo = foo
    self.bar = bar
    self.baz = baz
  
  def __repr__(self):
    return f"Test(foo={self.foo}, bar={self.bar}, baz={self.baz})"

my_inst = Test(2, 3, 4)
print(my_inst)

[fallen slate]

@surreal sun :white_check_mark: Your eval job has completed with return code 0.

Test(foo=2, bar=3, baz=4)

[paper echo]

Oof... at least you'll be contributing to open source! What kinds of warnings were they?

[white nexus]

96 warnings about accessing a deprecated parameter which will be removed in marshmallow 4

[calm flame]

有中国人在这里么

[verbal escarp]

actually, it's a good idea to do assert thing == eval(repr(thing)) in a test

iirc it's in the python docs

and it served me well, forces to parametrize things and have an actual representation of an object, which can help a lot in debugging

[paper echo]

Ouch

Hopefully that gets fixed

Hopefully it's all the same parameter and can be fixed in one place?

[white nexus]

its two deprecated parameters, used in multiple places by the library.

Looks to me that it would need to either add a multitude of ifs if it intends to continue supporting marshmallow 2, except I'm not sure that it still does.

[deep bramble]

How does a module like pygame or tkinter create graphical windows in the back? I mean how would you achieve that without these modules and perhaps create my own custom module?

[raven ridge]

It's a good idea, but not always possible. You can generally only do that for "value semantic" types, which aren't stateful. Consider something like the file object that you get back from open - it has state that can't be set by a constructor parameter, like its offset into the file, and so it's impossible to have a repr for it that fully explains its state in terms of how it was constructed. Only for value semantic types can you necessarily construct a new, equal, instance.

[deep bramble]

Is it just done with something like Cython with just C or is there a way to do it without any C extensions and purely use python?

[raven ridge]

if you're on Windows, at some point you would need to call into some DLLs provided by Windows to do the actual drawing. You could maybe do that with ctypes instead of Cython or C, but it's probably not fair to call ctypes "pure Python" anyway.

[unkempt rock]

Any one know about generative art in python

[verbal escarp]

true that, but there are a lot of cases where it could be done (for example packaging.Version) but it's not, probably because this wasn't known to the coder

[raven ridge]

yeah. I totally agree that it's a good practice for value semantic types.

[elder blade]

Oops I usually do <MyClass a=5 b=10>.

[white nexus]

my code now uses a combination of both that I effectively have stopped caring

[verbal escarp]

iirc there was a weird thing during module-building that relied on a <..> syntax for representations, which annoyed the heck out of me when i stumbled over it

[white nexus]

like

[verbal escarp]

can't recall the details, it had something to do with types

[white nexus]

attrs uses this syntax.

[white nexus]

but discord.py uses this syntax

so my code is effectively split between the two

that I have no more cares 😔

[surreal sun]

Agh, I was gonna make a library for contexttools, like functools but for how stuff is called syntactically, but i don't know what to add more than the one decorator I have

it's really weird stuff though

Like it can be used for esoteric crap

#internals-and-peps message
i found it

I can imagine how horrible this would be in prod

[white nexus]

hmmm

I just realised there's something to be said for when the majority of my new features have been figured out in #internals-and-peps 😬

[surreal sun]

Speaking of, what is so good about attrs btw, i've heard of it but I never understood it

[white nexus]

😔

[white nexus]

basically attrs is dataclasses but a lot lot lot better

[surreal sun]

Ah

Doesn't it make dunder methods easier

[white nexus]

yeah

[surreal sun]

No more

def __eq__(self, other):
  return self.value == other.value
...

[white nexus]

and does a lot more stuff

one sec

okay requested it to the docs

[white nexus]

yea, but the thing is the built in dataclasses module will do this too. Attrs has a lot of other features which make it extremely useful

for example, attrs has validation, converters, and many other things

so you could make a dataclass but also validate what goes in to a position very easily

[surreal sun]

OH wow

[white nexus]

!e ```py
from pprint import pprint
import attr
import logging

@attr.s(auto_attribs=True, slots=True, frozen=True)
class BotModeCfg:
"""
The three bot modes for the bot. Enabling some of these may enable other bot features.

`production` is used internally and is always True.
`develop` enables additonal features which are useful for bot developers.
`plugin_dev` enables additional commands which are useful when working with plugins.
"""

production: bool = attr.ib(
    default=True,
    converter=lambda _: True,
)
develop: bool = attr.ib(default=False)
plugin_dev: bool = attr.ib(default=False)

@attr.s(auto_attribs=True, slots=True)
class DevCfg:
"""Developer configuration. These values should not be changed unless you know what you're doing."""

mode: BotModeCfg = BotModeCfg()
log_level: int = attr.ib(default=logging.INFO)

@log_level.validator
def _log_level_validator(self, _: attr.Attribute, value: int) -> None:
    """Validate that log_level is within 0 to 50."""
    if value not in range(0, 50):
        raise ValueError("log_level must be an integer within 0 to 50.")

cfg = DevCfg()
pprint(cfg)

[fallen slate]

@white nexus :white_check_mark: Your eval job has completed with return code 0.

DevCfg(mode=BotModeCfg(production=True, develop=False, plugin_dev=False), log_level=20)

[white nexus]

That's a portion of the configuration system I made, slightly modified for snekbox

Most of this stuff could be done with dataclasses.

!d dataclasses

[fallen slate]

dataclasses

Source code: Lib/dataclasses.py

This module provides a decorator and functions for automatically adding generated special methods such as __init__() and __repr__() to user-defined classes. It was originally described in PEP 557.

The member variables to use in these generated methods are defined using PEP 526 type annotations. For example, this code...

[white nexus]

but the validators, converters, and other stuff cannot be done

and for the record

    production: bool = attr.ib(
        default=True,
        converter=lambda _: True,
    )

I am aware this is hacky. I don't care enough to make it not hacky lol

making it just always True was the easiest, least hacky solution, since the entire reason the production key exists is from a previous hack, but left in to make everything so much easier

@surreal sun ^

[surreal sun]

Hm

[verbal escarp]

class boolean:
    def __init__(self, x):
        if isinstance(x, str):
            try:
                self.x = bool(literal_eval(x))
            except ValueError:
                raise ConfigurationError(f"{x} is not a Value of True or False!")
        if isinstance(x, bool):
            self.x = x

    def __str__(self):
        return str(self.x)

    def __repr__(self):
        return repr(self.x)

    def __bool__(self):
        return self.x


@attr.s(
    auto_attribs=True,
    on_setattr=attr.setters.convert,
    field_transformer=lambda cls, fields: [
        field.evolve(converter=field.type) for field in fields
    ],
)
class Config:
    # "False" is a non-empty string -> True :|
    first_start: boolean = True
    database: str = "watnu.sqlite"
    lucky_num: int = 1

my take on the subject ^^

[white nexus]

wdym

[verbal escarp]

i also made a config with attrs

[white nexus]

ah nice

what isn't listed is I'm also using marshmallow for my configuration 😅

although I'm not sure that's really needed anymore....

[verbal escarp]

the config itself is a pure text file with key/values, only str

the conversion/validation is done just in that class

[white nexus]

no, wait, I'm using marshmallow to load the loaded toml dict into the attr classes.

and marshmallow is extremely nice for this feature 😛

if it's not obvious, its being able to exclude secure values from a dump

which is very useful for when saving the rest of the configuration to a database

although

ugh

I should look at how to use sql alchemy and get started in the world of databases

wdym?

But how would you iterate over the keys only of a dict?

TBH

I think the current state of the matter makes sense

for k in dict

for k in list

🙂

say you have a list, and wanted to change it to a dict, because you need the values somewhere. Most interactions with a list stay valid for a dict

its pretty easy to then just change the definition and all current code does not have to change

😂

[sand goblet]

I think that that would be the only time python does something like that

[verbal escarp]

i wondered that myself, many times

damn

that was the wrong msg

that one

it would be nicely symmetrical with dict comprehensions

[flat gazelle]

this is impossible since a key may be a tuple

[verbal escarp]

little announcement: we just released https://pypi.org/project/justuse/0.5.0/

please report any bugs on the tracker, that would be very useful

.oO(wondering why the html on pypi is bugged)

i still wouldn't recommend using it in production, but auto-install should work for most cases now

[haughty geyser]

im new to python on visual studio. can someone help me on a 1 to 1 in dms?

[verbal escarp]

any case that doesn't work and is reported on the tracker will be turned into a testcase to make sure it will be covered in the future

[surreal sun]

I stumbled upon this
https://github.com/borzunov/plusplus/blob/main/src/plusplus/

[onyx sparrow]

interesting

[surreal sun]

Quite interesting as to what you can do with bytecode manipulation

[lusty scroll]

using an import hook?

[surreal sun]

wdym?

[lusty scroll]

the bytecode hack gets installed by adding a meta_path finder it looks like?

[surreal sun]

Never heard of what a meta_path finder is

Could you explain as to what it is?

[lusty scroll]

that's pretty smart. I wonder if other combinations would be possible, like

a = ~~b
b = --a

[verbal escarp]

i guess any unitary op could be used

but i don't like abusing those just for the sake of it

[lusty scroll]

sys.meta_path works kind of like sys.path allows you to basically intercept the importing of any module to perform arbitrary manipulation, like changing bytecode or source code, etc., like for rewriting modules at load time (and independent of the filesystem), is my understanding

[surreal sun]

Ahh

Thta's cool

[lusty scroll]

simple one to enable importing from a dict storing paths and sources
https://github.com/ropez/pytest/blob/master/_pytest/standalonetemplate.py

[white nexus]

sigh

this is def the channel for this 😔

How to sort a dictionary?

[surreal sun]

https://stackoverflow.com/a/613218/16780882

[white nexus]

damn

[surreal sun]

Would anyone know as to why it is telling me that i need a type integer, when these are integers?

from types import CodeType
def patch_bytecode(change_to: str, to_replace: str):
    def decorator(function):
        nonlocal change_to, to_replace
        bytes_in_code = function.__code__.co_code
        to_replace = bytes_in_code.index(dis.opmap[to_replace].to_bytes(1, byteorder="little"))
        change_to = [dis.opmap[bytecode].to_bytes(1, byteorder="little") for bytecode in change_to.split('\n')]
        new_co_code = bytes_in_code[to_replace:]
        for bytecode in change_to:
            new_co_code += bytecode
        if to_replace + 1 < len(bytes_in_code):
            new_co_code += bytecode[:to_replace+1]
        bytes_in_code = new_co_code
        fn_code = function.__code__
        function.__code__ = CodeType(fn_code.co_argcount,
                                     fn_code.co_kwonlyargcount,
                                     fn_code.co_nlocals,
                                     fn_code.co_stacksize,
                                     fn_code.co_flags,
                                     bytes_in_code,
                                     fn_code.co_consts,
                                     fn_code.co_names,
                                     fn_code.co_varnames,
                                     fn_code.co_filename,
                                     fn_code.co_name,
                                     fn_code.co_firstlineno,
                                     fn_code.co_lnotab,
                                     fn_code.co_freevars,
                                     fn_code.co_cellvars,
                                     )

        def wrapper(*args, **kwargs):
            return function(*args, **kwargs)
        return wrapper
    return decorator

Error is where it says function.__code__ = CodeType

[white nexus]

12 year old question and no one did it recursively

[surreal sun]

Oop

[peak spoke]

You're missing an int arg at the start

[surreal sun]

Ah, does it matter as to what the argument is at the start?

Or do you mean that fn_code.co_argcount needs to be an integer

because it already is iirc

[peak spoke]

No, you have 5 args before your bytes_in_code while it expects one more

[surreal sun]

ahh

[peak spoke]

Look at the help of the type

[white nexus]

got it

    def sort_dict(d: dict) -> dict:
        """Takes a dict and sorts it, recursively."""
        sorted_dict = {x[0]: x[1] for x in sorted(d.items(), key=lambda e: e[0])}
        for k, v in d.items():
            if isinstance(v, dict):
                sorted_dict[k] = sort_dict(v)
                continue
        return sorted_dict

[surreal sun]

Yay, I finished being able to manipulate bytecode

[white nexus]

yessssssssssssssssssssssssssssssssssss

it exports to ```yaml
answer: 42
bot:
prefix: '?'
colours:
base_embed_color: '7506394'
dev:
log_level: 20
mode:
develop: false
plugin_dev: false
production: true

{
    "answer": 42,
    "bot": {
        "prefix": "?"
    },
    "colours": {
        "base_embed_color": "7506394"
    },
    "dev": {
        "log_level": 20,
        "mode": {
            "develop": false,
            "plugin_dev": false,
            "production": true
        }
    }
}
``````toml
# This is an autogenerated TOML document.
# Directly run the config.py file to generate.

answer = 42

[bot]
prefix = "?"

[colours]
base_embed_color = "7506394"

[dev]
log_level = 20

[dev.mode]
develop = false
plugin_dev = false
production = true
```

the question is, do I add support for loading yaml files? 😂

[sacred quarry]

I don't know if this is what you're looking for (nor do I even know why I'm here) but...

!pypi pyyaml

[fallen slate]

PyYAML v5.4.1

YAML parser and emitter for Python

[white nexus]

ye, already using that

I only added dump support for json and yaml for fun temporarily

    # toml
    with open(pathlib.Path(__file__).parent / "default_config.toml", "w") as f:
        atoml.dump(doc, f)

    # json
    import json

    with open(pathlib.Path(__file__).parent / "default_config.json", "w") as f:
        json.dump(dump, f, sort_keys=True, indent=4)

    # yaml
    import yaml

    with open(pathlib.Path(__file__).parent / "default_config.yaml", "w") as f:
        yaml.dump(dump, f, sort_keys=True, indent=4)

but I do like the yaml file enough that I may keep it around lol

[verbal escarp]

i hate yaml

it looks so innocent

but it's a bitch if you get to know it better

[raven ridge]

toml is like a saner yaml

[surreal sun]

You know what else isn't sane? Messing with python bytecode

My exit code was some long as hell hexadecimal numbe

r

I think I screwed with it a bit too much 😳

[paper echo]

Multi line strings in yaml are great

I know there's weird stuff like anchors and putting multiple "documents" in the same file

[white nexus]

cc @paper echo of this message, which format looks better?
#internals-and-peps message
I actually think that the toml is a bit too verbose

[surreal sun]

Do you all know about the contextdispatch thing I was talking about yesterday?

Yeah.. it's pretty cursed

I can overload and and or as long as I do str(class_instance) and catch the syntactic context

[white nexus]

I think

I want to

I

[surreal sun]

Code:

Result

this is quite cursed

[swift imp]

can someone explain to me what this context dispatch is

[surreal sun]

It's a custom decorator I created

How it works:
#ot0-psvm’s-eternal-disapproval message

[swift imp]

right but what its supposed to do, I've been reading this for past 2 or 3 days

[surreal sun]

kind of function overloading,

based off how it is called, it can call different functions

[swift imp]

Yes I understand dispatching, I've done state and type based using decorators and descriptors but what does the context mean, how is it different than state or type based?

[surreal sun]

The context means as to how I called it, for example if I had
It essentially gets the top value on the call stack
example:

!e

from inspect import stack

def foo():
  print(stack()[-1])

foo()

[fallen slate]

@surreal sun :white_check_mark: Your eval job has completed with return code 0.

FrameInfo(frame=<frame at 0x7fe1ae818ac0, file '<string>', line 6, code <module>>, filename='<string>', lineno=6, function='<module>', code_context=None, index=None)

[surreal sun]

weird, on snekbox it says code_context=None

[peak spoke]

It has no source file to use

[surreal sun]

Ah

That is why

!e

from dis import opname
from inspect import stack


def foo():
  print(opname[stack()[-1].frame.f_lasti])

foo()

[fallen slate]

@surreal sun :white_check_mark: Your eval job has completed with return code 0.

<34>

[surreal sun]

That's.. interesting

[raven ridge]

I'll just leave this here: https://www.arp242.net/yaml-config.html

[white nexus]

👀

I may switch my code to use strictyaml over pyyaml

[summer epoch]

Is there any reason why cpython decided to use a stack based runtime over a register based one? I have found some answers online but none specific to python itself.

I have played around with both vm models and it seems as though a register based runtime would be much more efficient in the majority of scenarios? take basic arithmetic for example, we can simply perform a single operation between two registers, rather than popping 2 values off of the stack and pushing it back on.

I know that python has a binary_add opcode, but this still seems as though it does much more work than a register based one would require

[empty kite]

I know there are ways to constrain TypeVars, but is there a way to express "any type except NoneType"? Any includes NoneType

[grave jolt]

No, there isn't.

You might be able to write/find a mypy extension for that, but don't take my word for it

Why do you need that?

[empty kite]

I was afraid there wasn't. I don't want to have to deal with None in my user facing library code, since it wouldn't make sense to supply it anyways. I don't really need it, since the type hints don't do much, but I'd like to have it as documentation

Seems like a pretty big oversight not to include a non-nullable type in the typing modules

[grave jolt]

@empty kite Why do you need a TypeVar that's guaranteed to not be None? If a generic type is totally generic, it works with any type as a parameter, including None and Literal[42]. Can you give an example of where you want it?

[empty kite]

Where storing None would not be unsound, but a domain error

[grave jolt]

Can you give an example?

[empty kite]

Unfortunately, I can't

[grave jolt]

well, I worked with type hints for a while, and I can't think of one either 🙂

I've seen some people ask for OneType except ParticularSubtype in this situation: they want to accept Iterable[str], but don't want to accept str (it is an iterable of strings, but it's probably a programmer error if they do that)

that seems reasonable, and TypeScript actually has this feature

in TypeScript IIRC it's useful when you have a tagged union, and you want to exclude some particular variant

[empty kite]

Well, you could convert the string, but possibly it makes sense to alert the user instead. The except keyword seems weird in this context. as is used in a few different places though, so ymmv

[grave jolt]

oh, I didn't mean exactly this syntax, that's just to convey the idea

[empty kite]

I see. Something like that could be useful

[white nexus]

I'm not sure if this is cursed or genius

    states = {
        "The yaml library is not installed.": yaml is not None,
        "The provided yaml config path does not exist.": path.exists(),
        "The provided yaml config file is not a readable file.": path.is_file(),
    }
    if not all(states.values()):
        errors = ""
        for key, value in states.items():
            if not value:
                errors += key + "\n"
        raise CfgLoadError(errors)

[boreal umbra]

@white nexus I need for it to be fewer, longer statements

You can easily make errors in one statement and raise an exception if it's not an empty string.

I also don't agree with this dict usage. They're using a dict as a list of tuples.

[empty kite]

Aside from the dict stuff, here's the "fewer, longer statements" version:

if errors := "\n".join(key for key, value in states.items() if not value):
    raise CfgLoadError(errors)

[boreal umbra]

I like where you're going with this. I need for states.items() to be a list of tuples literal

[white nexus]

that is me, I wrote that 👀

[empty kite]

Sure:

states = [
    ("The yaml library is not installed.", yaml is not None),
    ("The provided yaml config path does not exist.", path.exists()),
    ("The provided yaml config file is not a readable file.", path.is_file()),
]
if errors := "\n".join(msg for msg, check in states if not check):
    raise CfgLoadError(errors)

[boreal umbra]

Yes but I want states to be a literal in the if statement. No name assigned to it.

And I want the text of the error message to be red

[white nexus]

you mind if I use this in my MIT licensed code? 👀

[empty kite]

I see, we're golfing now

[white nexus]

(see edit if you give consent)

[empty kite]

I'd be humbled

[boreal umbra]

arl asked if it was cursed and I'm trying to make sure that it is

[surreal sun]

!e

from forbiddenfruit import curse

def to_list(self):
    return [(key, value) for key, value in self.items()]

curse(dict, "to_list", to_list)

print({"hi": 2, "bye": 1}.to_list())

[fallen slate]

@surreal sun :white_check_mark: Your eval job has completed with return code 0.

[('hi', 2), ('bye', 1)]

[white nexus]

lol

[white nexus]

added, have no way of knowing if it works yet-- I've been busy refactoring this code for the past hours now since it was untestable in its previous state

now its unusable, too

[boreal umbra]

You could just pass dict items to list

No for loop or list comp

[surreal sun]

Oh, wait, lmao

[fallen slate]

@pliant tusk :white_check_mark: Your eval job has completed with return code 0.

None

[surreal sun]

!e ```py
from dis import dis

dis("""
def foo():
try: yield
except Exception as e:
print(e)
yield

f=foo()
f.send(None)
f.throw(SyntaxError)""")```

[fallen slate]

@surreal sun :white_check_mark: Your eval job has completed with return code 0.

001 |   2           0 LOAD_CONST               0 (<code object foo at 0x7f21b8061920, file "<dis>", line 2>)
002 |               2 LOAD_CONST               1 ('foo')
003 |               4 MAKE_FUNCTION            0
004 |               6 STORE_NAME               0 (foo)
005 | 
006 |   8           8 LOAD_NAME                0 (foo)
007 |              10 CALL_FUNCTION            0
008 |              12 STORE_NAME               1 (f)
009 | 
010 |   9          14 LOAD_NAME                1 (f)
011 |              16 LOAD_METHOD              2 (send)
... (truncated - too many lines)

Full output: https://paste.pythondiscord.com/bilixavini.txt?noredirect

[surreal sun]

My guess (don’t take my word for granted) is that you’re setting the generator to None now, within try after sending None into the yield value, and NoneType has no attribute throw, so it just ignores it, but that might be wrong considering e isn’t even printed

[sand goblet]

In python source code, what is all this stuff at the end of the code for an object?
https://github.com/python/cpython/blob/fcbf9b176b1190301c760a921601c6488ef8b070/Modules/itertoolsmodule.c#L1949

[fallen slate]

Modules/itertoolsmodule.c line 1949

static PyTypeObject starmap_type = {```

[sand goblet]

And are there any guides on how to read python source code?

It seems really confusing

[sacred yew]

relevant docs: https://docs.python.org/3/c-api/typeobj.html

[sand goblet]

Ok thank you, it seems like that answers it

[raven ridge]

https://docs.python.org/3/c-api/intro.html would be a good starting point, if you haven't read that.

Actually, better yet, https://docs.python.org/3/extending/index.html

The former is more reference docs, the latter is more tutorial

[sand goblet]

Good, thank you guys for finding these. They look really helpful

[raw trench]

Hello?

Am new on

Discord

[burnt escarp]

h

[foggy flower]

can someone help me in #help-grapes

[lusty scroll]

if you put your question in #discord-bots i have an idea

[foggy flower]

done

[paper echo]

I think YAML looks better if that's the level of nesting that you want. JSON is for machines, not people.

Sigh... i tried. https://bugs.python.org/issue45027

@halcyon trail CC because we were talking about this a while ago

[mint forge]

Oh, was that you who opened the issue?

[paper echo]

Yeah, not sure if I came across wrong or it just wasn't meant to be

I was surprised to find so much resistance in the first place

It's fine, I'll throw it on pypi and pitch it as an alternative to loguru or something

I understand the resistance to "adding stuff"

[halcyon trail]

WHO SUMMONS ME

ah yes

@paper echo to be completely honest I mostly agree with Vinay

[paper echo]

interesting

why? just too many different "ways to do it" in the stdlib?

[halcyon trail]

a) it's not a common use case
b) it can be done easily with a utility function on top of the existing functionality

I think that's basically the bottom line

if something is a common use case, then the stdlib should provide a direct convenient API, yes. If it's not, then the stdlib, or any lib, cannot support everything, obviously, so then it's just a question of whether it makes it possible for users to do what they want, because impossible of course would be bad.

In this case, I don't think this is a particularly common use case. getLogger(...).... is already the defacto way to do more complex manipulations of logger configuration; you can get any logger, do anything with that API.
logging already provides two convenience APIs on top of that, basicConfig, and this dictConfig which I'd never even heard of until now.
two convenience APIs sounds like plenty? Especially considering that basicConfig is the 95% case.
I'm not saying what you want isn't valid, but my guess is that if we poll enough python developers and ask them wha tutility funcitons they have for dealing with python logging, you'll find dozens and dozens of examples just as compelling as yours
And it's not reasonable to have them all in logging

I'm actually surprised that dictConfig even exists at all, but maybe they thought it would be useful to have a "standardized" way to do almost arbitrary configurations of logging from a config file.

[gleaming rover]

🥴 TIL of dictConfig

[paper echo]

b) it can be done easily with a utility function on top of the existing functionality
i guess this is the part that bothers me - this is redundant boilerplate in 90% of applications

and yeah, dictconfig is just really ugly

i proposed this for the same reason that we have str.removeprefix and :=

you don't "need" it -- but the alternative is copying and pasting the same crap in 1000 places

although a more concise dictConfig would also be pretty nice...

maybe whatever i put on pypi would also support that

another thing for the backlog i guess

[halcyon trail]

what do you mean, redundant boilerplate in 90% of applications? what makes you think 90% of applications are doing this?

[paper echo]

almost every python app i've worked on has had something like

def configure_logging():
    logger = logging.getLogger('thing1')
    logger.handlers = []
    handler = logging.StreamHandler()
    formatter = logging.Formatter(logfmt_warning)
    handler.setFormatter(formatter)
    logger.addHandler(handler)
    logger.setLevel(logging.WARNING)

    logger = logging.getLogger('thing2')
    logger.handlers = []
    handler = logging.StreamHandler()
    formatter = logging.Formatter(logfmt_info)
    handler.setFormatter(formatter)
    logger.addHandler(handler)
    logger.setLevel(logging.INFO)

yes, dictconfig would help

[halcyon trail]

I've almost never seen that 🤷‍♂️

[paper echo]

what do you see? basicConfig and leaving it as-is?

or basicconfig + selectively disabling libraries that log stuff you don't want to see?

[halcyon trail]

pretty much. Only time I personally ever needed to configure a higher level logger is actually a particular logger that spews out way too much stuff

yeah, pretty much, basicConfig plus disabling libraries.

But even then, usuallyw hat happens is that the libraries you want to disable are "repeat offenders", like paramiko, so typically you just end up with your own utility function, anyway, that does some of teh logging related stuff that you want, including disabling noisy libraries

maybe part of my point is that everyone who's using logging uses it at least a little different, and everybody ends up with some utility functions anyway

like having a utility function that takes argparse's parsed arguments and sets up the logging based on that, because most people are going to have some set of arguments that recur over and over

[paper echo]

perhaps basicConfig isn't the right place to add this, then. but i do think having some interface like

add_new_handler(
    loggername_or_logger,
    level_or_levelname_or_levelnum=...,
    format_or_formatter=...,
    handler_or_stream_or_filename=...,
)

would save a lot of confusion, and redundancy/boilerplate

otherwise you force people to implement ad-hoc versions of of that function over and over and over

so that's what i'm going to put on pypi, even though nobody in their right mind would pull in a dep for something so trivial 😛

an alterantive interface to the fileconfig/dictconfig thing might be more valuable

[halcyon trail]

Well, add_new_handler(loggername, ...) isn't very different from getLogger(loggername).addHandler(...)

[paper echo]

getLogger(loggername).addHandler(StreamHandler().setFormatter(Formatter('...')))

and then all the if/else stuff on top of it

actually does setFormatter even return the handler? it probably returns None

[halcyon trail]

not sure what you mean by if/else.

but yes, I think that part could be made more convenient

[paper echo]

...setFormatter(Formatter(format) if isinstance(format, str) else format)

[halcyon trail]

Gotcha

Yeah, adding a member function to Logger that would handle all this automatically would be nice, IMHO

or just a factory function that creates the handler