internals-and-peps

[acoustic crater]

ah

[raven ridge]

it's pretty much required, if you're designing a class without knowing whether or not users will inherit from it.

[acoustic crater]

sry I've been breaking python for a few hours I'm not ready to joke about it

was uninspired all day then I learned what #esoteric-python is for

[boreal umbra]

For inits. But other times too?

[raven ridge]

for any method where you call super() to delegate to the next class in the MRO, at least.

And, if you want the next class to be able to take different arguments for that method.

[boreal umbra]

Speaking of

I made a class method constructor that calls a different class method constructor for the superclass

And then changed the class of the instance that gets made. is that bad?

I'm not going to change it because I like it.

[raven ridge]

like, you changed obj.__class__ before returning it?

[boreal umbra]

Also the subclass doesn't obey the liskov principle

[raven ridge]

obeying LSP is pretty much the single most important thing when using inheritance.

[boreal umbra]

I might go back in later and make it so they're two separate classes, but give the other class references to some of the methods from the first.

[proud fern]

how to install flaskr ?

[acoustic crater]

I wrote a subclass that sometimes redefines its own class but that's cuz it's designed to built itself based on kwargs and superclass and derivatives are designed to interact with the same class specifically

it obeys LSP tho

...without kwargs it's just the superclass

and with em it just doesn't let you add em together

[gritty geode]

Advenced discussion lol I'm a newbie

[glad token]

For a terrible time: https://perlmonks.org/?node=Obfuscated Code

[acoustic crater]

lol I'm more interested in doing stupid things that take advantage of reflection than that

[unkempt rock]

Hey everyone.. I’m student IT program in Denmark and looking the teams members for Registration for MathWorks Minidrone Competition at Nordics and Baltics 2021..

[hollow garden]

one last thing

isnt pythons syntactical growth a bit worrying the past year?

[hexed maple]

Yeah

[flat gazelle]

ye, it is becoming somewhat bloated recently. things like :=, however neat they may be, do desimplify the language quite a bit

[hollow garden]

like there was a shitton amount of syntax added to python

[hexed maple]

So many changes

3.10 rht

[hollow garden]

and even more shitty pep proposals seems like are going to be accepted

e. g pep 637

the fuck is that?

[feral cedar]

!pep 637

[fallen slate]

**PEP 637 - Support for indexing with keyword arguments**

Link

Status

Draft

Python-Version

3.10

Created

24-Aug-2020

Type

Standards Track

[oblique crystal]

idk, you are not obliged to learn/use it right away

[feral cedar]

how does that even work

[flat gazelle]

oddly

it is really a weird pep

[hollow garden]

the sad thing is more than half of the community wants it

[oblique crystal]

like you can still learn language as we used to, no

[flat gazelle]

it is an issue, since you will see it in code used

[hollow garden]

we are slowly getting away from pythons intended philosophy

[flat gazelle]

and so you have to learn it

saying don't use a feature if you don't like it doesn't work, since other people will use it and you need to understand other people code

[hollow garden]

!pep 20

[fallen slate]

**PEP 20 - The Zen of Python**

Link

Status

Active

Created

19-Aug-2004

Type

Informational

[hollow garden]

thats why new syntax is supposed to be bad too

those are not things supposed to be in python

[flat gazelle]

like, looking at a simple language like elm, that language removes features with every new version

I feel like python is trying to compete with all those new fancy languages with feature lists over several pages and thereby losing what makes python nice in the first place

[hexed maple]

Has anyone ever even used the walrus ?

[oblique crystal]

about 637, this I would very much like to have 🙂

[hexed maple]

I don't

[flat gazelle]

I used it like twice

there is a proposal for macros as well

they are a really powerful tool that would allow a lot of previously impossible APIs

[hollow garden]

that is really confusing to me

and i would be even more confused with the new syntax

[hexed maple]

Same

[oblique crystal]

I am 99% sure that it's way it works in R which is considered for data-viz/analysis to be better than pandas 🙂

[hexed maple]

Is that assignment or equi check ?

[oblique crystal]

equal check

well, filtering

[hexed maple]

Ahh

Ol

Ok

[oblique crystal]

df[df['x'] == 1] means select rows of df where column 'x'=1

[hexed maple]

But still I like the former one

Much more readable

[hollow garden]

^

python must be readable and explicit

[flat gazelle]

yeah, these are all useful things for some specific libraries. But the issue is, people will use and overuse it. People like using fancy new things

[hollow garden]

speaking of library specific syntax, the fuck is @?

i just learned it existed last week

[flat gazelle]

it is an operator

[hollow garden]

for matrix multiplication only for numpy arrays?

[oblique crystal]

df[(df['x'] == 1 ) & (df['y'] == some_func(50))]

for me it becomes to much boilerplate

[flat gazelle]

it is meant for matmul

it uses the __matmul__ dunder

[hollow garden]

however someone who isnt used to the new syntax can understand it

[flat gazelle]

numpy arrays do use it for matrix multiplication

== being broadcasted is not really intuitive in any way

you expect == to result in True or False

not a whole series

[oblique crystal]

perhaps but I saw a lot of people, including myself who often attempt

df[x == 1 ) & (y == some_func(50))]

[hollow garden]

python is going to be more complex than rust at this pace

[hexed maple]

Py is kinda simpimg for pd community I guess

[hollow garden]

^

[flat gazelle]

maybe its trying to not die to Julia

in that domain at least

[hollow garden]

it doesnt seem like its going to die anytime soon

[unkempt rock]

is the walrus operator controversial?

[hexed maple]

No

[hollow garden]

yes

[unkempt rock]

what is so bad about it?

[hexed maple]

Readability

[flat gazelle]

it is quite confusing and not all that useful

[hollow garden]

readability for beginners

[unkempt rock]

i love that i can do print(a := 'hello') without another line

[flat gazelle]

[a for a in (v:=range(10))]

[unkempt rock]

and use a anywhee else

[flat gazelle]

take a guess at what this does

[hexed maple]

Another line adds a lot more readability

[unkempt rock]

what is a for a in range?

i haven't learned that yet

[flat gazelle]

ah, you didn't do comprehensions yet

[hollow garden]

list comprehension

[unkempt rock]

oh right

now i remember

[hollow garden]

lol

[hexed maple]

Hehe

[hollow garden]

what does it do by the way?

[flat gazelle]

try it out, you will probably be surprised

[hollow garden]

im on mobile lol

[unkempt rock]

so basically, it's iterating through the range 10, which is assigned to V

right @flat gazelle ?

[flat gazelle]

you would think so

[hollow garden]

thats what it is supposed to be

[flat gazelle]

!e [a for a in (v:=range(10))]

[fallen slate]

@flat gazelle :x: Your eval job has completed with return code 1.

001 |   File "<string>", line 1
002 | SyntaxError: assignment expression cannot be used in a comprehension iterable expression

[flat gazelle]

but actually, turns out it makes the scopes hard to resolve and so it is forbidden

[hollow garden]

lol

[unkempt rock]

hmm?

[flat gazelle]

and if you have to special case a feature to forbid it in certain contexts, is it really a simple and non confusing feature

I don't even think this is the only case

[unkempt rock]

could you write this without the walrus operator?

[flat gazelle]

v = range(10)
[a for a in v]
``` works just fine

[unkempt rock]

ok

makr sense

[flat gazelle]

so yeah, adding features, even if they aren't exactly bad, is concerning

[hexed maple]

Like f strings
There is a forbidden in that too

[flat gazelle]

it is possible in those, you just have to do f'{(a:=4)}'

[unkempt rock]

any actual useful use cases for walrus operator?

[flat gazelle]

while loops

while a := read(1024):
    print(a, end='')

where read is some theoretical function that reads some amount of data

[unkempt rock]

ok

[flat gazelle]

you can also do this with iter

[hollow garden]

the fuck

[pliant tusk]

@flat gazelle the reason for the scopes issue is because the inside of list comprehensions are compiled as separate code objects. And the assignment expression would end up inside that for loop code object

And that raises the question of where v would be defined after the comprehension

[flat gazelle]

yeah, until := it wasn't an issue since you couldn't create variables in expressions

[unkempt rock]

why were f-strings even made? .format is pretty good

[hexed maple]

they are awesome

[hollow garden]

well they are good i guess

[flat gazelle]

because they are nicer than .format

[hexed maple]

But u can't use * in them

[unkempt rock]

nicer?

in what way?

[hollow garden]

if its confusing it is as confusing as .format

[flat gazelle]

you don't have to state the name of everything 3 times

[hexed maple]

Hehe

[flat gazelle]

'{name}'.format(name=name)

vs f'{name}'

[unkempt rock]

name=name?

[hexed maple]

Just have a look at float precision

[flat gazelle]

a variable name needs to interpolated into that point

positionals aren't useful if you have multiple things to put in, since they get hard to parse

[pliant tusk]

i have noticed an interesting thing with fstrings. ```py

f'\{1}'
'\1'
f'{1}'
'\1'

[true ridge]

features like :=, f-strings, matrix operator had very controversial periods of discussions. Though for the new syntaxes, like PEP 637 suddenly everyone agrees or people just don't want to comment and deal with mailing lists

[hexed maple]

That's a good SQL injection check lol

[peak spoke]

>>> import warnings; warnings.resetwarnings()
>>> f'\{1}'
<stdin>:1: DeprecationWarning: invalid escape sequence \{
'\\1'

[hexed maple]

I got one too
What should be the output of
f'\1'

[flat gazelle]

I think it would yell at you for incomplete octal escape

oh wait, it just takes the number

[peak spoke]

Mailings lists are nice for the core devs etc. around it but I don't think the wide community will contribute there much

[pliant tusk]

in 3.6 that would output '\\{1'

[hexed maple]

!e
f'\1'

[fallen slate]

You are not allowed to use that command here. Please use the #bot-commands channel instead.

[hexed maple]

Someone run it

[pliant tusk]

>>> f'\1'
'\x01'
>>> ```

[hexed maple]

Why ??

[pliant tusk]

its an octal escape

[hexed maple]

I gotta escape yeah

[peak spoke]

https://docs.python.org/3/reference/lexical_analysis.html#index-22

[pliant tusk]

so \0 - \a convert automatically

[hexed maple]

Yeah got it 👍

[peak spoke]

I guess a bit unexpected but what it does currently seems to make more sense to me, if we forget that python should be warning against the usage

[unkempt rock]

why is python so whitespace sensitive and tab sensitive? if it used braces and semicolons we wouldn't have to worry about indentation

[visual shadow]

Run the command from __future__ import braces in your code

[unkempt rock]

i know the import

but why?

why did guido decide to make it tab sensitive?

[astral gazelle]

Indentation is a non issue tho

[visual shadow]

i think it "has" to be fixed throughout the file, that's the only real criteria python imposes

PEP8 recommends 4 spaces though, because mixing code with tabs and spaces got really messy really fast

there was a time when python wouldnt raise an issue about it

[unkempt rock]

it requires 4 spaces right? or can we do 2?

[spark magnet]

Python doesn't require a particular indentation. PEP 8 recommends 4 spaces.

[unkempt rock]

oh nice

[spark magnet]

but spaces are much better than tabs 🙂

[unkempt rock]

i i was lazy, could I do 1 space?

[visual shadow]

i'd encourage 4 simply because it helps others who read your code. once you get used to it it's a breeze

[spark magnet]

i use 1 space in the REPL.

[visual shadow]

yes. but 1 space makes it "very hard" to visually see indentation in big codebases

[unkempt rock]

i know, just asking

[visual shadow]

yep, python can have arbitrary spacing, as long as its consistent

er. thats a weird sentence. as long as you use the same "number of spaces", it doesnt matter "what" the number is to python

[unkempt rock]

ok

can I have 4 spaces for one if statement?, and 8 for another?

[visual shadow]

i think so off the top of my head, but play around with your editor/ide of choice for sure

[unkempt rock]

ok

[limpid forum]

just use an IDE that automatically converts tabs to spaces. press tab once, get 4 spaces - easy

[unkempt rock]

i know

[grave jolt]

!e

if 1:
 if 2:
  if 3:
    if 4:
        if 8:
                if 16:
                                if 32:
                                                                print("yep")

[fallen slate]

@grave jolt :white_check_mark: Your eval job has completed with return code 0.

yep

[unkempt rock]

whoa

[peak spoke]

Python only cares about one block being consistent

[flat gazelle]

you can even mix spaces and tabs

[spark magnet]

That is a great demonstration 🙂 Reminds me of my favorite stack overflow answer: https://stackoverflow.com/q/3949422/14343

[grave jolt]

lol

[raven ridge]

If the question is limited to how many pixels wide the character is, the answer should be "M" in most fonts.

Not that it was a good question, but that's probably the most correct answer 🙂

[grave jolt]

@spark magnet Can you somehow construct a clever answer that will point to the right character depending on the font?

[limpid forum]

Or W? I think M usually is more vertical than W

[limpid forum]

I haven't looked into the link, but generally you would write the character onto a bitmap and find the enclosing rectangle. This would work for any font and any character, as it's just "write and check" but automated

Hm, instead of bitmaps, one of the answers in that question uses html div without size and check the calculated size when changing contents via js. Similar thing

[undone hare]

monospaced fonts

[acoustic crater]

When I automated gta horse race betting and tracking odds I just made a pixel ruler function that detected light/dark pixels and manually counted pixels in screenshots to determine the logic

certain characters always had certain height/width of light/dark

[white nexus]

All of 2 and before, yeah?

[limpid marten]

This was an enjoyable read.

[rose stream]

Is there a way to make like a typescript interface in python/

[sacred yew]

typing.Protocol?

[rose stream]

Interesting, this is what I was looking for.

Thanks for the help!

Damn, it only works if your passing in a class

[grave jolt]

1. Union[str, Any] is exactly the same as Any.

2. Do you really want the function to accept an Optional[int] which is Union[int, None], or do you want it to accept an optional argument?

3. You can make a type alias:

MyCallback = Callable[[str, str, str, Optional[int]], WebElement]

4. Those positional arguments are very cryptic, and you can't specify an optional argument like that. Instead, you can make a protocol:

class MyProtocol(MyCallback):
    def __call__(self, attribute: str, type: str, condition: str, wait: int = ...) -> WebElement:
        ...

5. You might consider using an Enum or Literal if you want to allow only specific strings. It's similar to TypeScript's literal types.

AttrType = Literal["id", "name", "thing", "stuff"]

class AttrType(str, Enum): 
    id = "id"
    name = "name"
    thing = "thing"

(if you're coming from TypeScript, you might be disappointed about some things, like type inference)

If you're using mypy, consider also trying Pyright (or Pylance, if you're using VSCode)

[rose stream]

Many thanks for the detailed response!
I'm using pylance

1.) You're right, It's just a placeholder right now until I find the second type to use

2.) I wanted it as an optional argument, I guess it uses the parents kwarg which is always an int

[grave jolt]

protocol doesn't require you to make a custom callable class, it describes a plain function as well

[rose stream]

You're right. 3 && 4 seems like a much cleaner implementation - So handle_wait_factory would be

def handle_wait_factory(browser: Webdriver, *args) -> MyProtocol):
  pass

Damn. That looks so much cleaner

[flat gazelle]

@buoyant dirge we will not help with exams

[buoyant dirge]

please dont assume such things

with little to absolutely zero knowledge

it's not for an exam

[flat gazelle]

well, regardless, it is not suited for this channel.

[round lion]

for those of you using CI/CD in Python development, more geared towards utilities than web applications, what's usually in your CI pipeline? At the moment I'm only doing linting checks and unit testing with pytest. I'm thinking doc string linting, code coverage, and code complexity checks may be useful but wanted to get a second opinion. Could also do static type checking with mypy but that seems overkill for Python.

[spark magnet]

@round lion why is mypy overkill? It's linting like pylint is.

[round lion]

static typing seems overkill for Python development tbh

[spark magnet]

@round lion definitely coverage is good, but I am biased. 🙂

if you use type hints, then mypy in CI makes sense.

[round lion]

yeah, I don't think I'll convince devs to do type hints even though it'd make their doc string generations quicker

[spark magnet]

do you measure coverage?

[round lion]

somewhat. For some we do, others no

I'm wanting to standardize, so I'm trying to figure out how deep in CI I throw people/myself

linting, unit testing, and code coverage seem easiest since you can find auto formatters for any IDE depending on your linting preference. Doc strings and type hints seem like the harder ones

[raven ridge]

My linting is usually flake8 plus a Sphinx dry-run, sometimes coverage with a 100% threshold, usually mypy on an installed copy of the package - though I do mostly libraries, where mypy has a bit more value than for applications

[round lion]

ooo, never heard of sphinx. We use flake8 for our linting

although I think our flake8 config could probably use some work. We use black as our formatter, and I don't believe our flake8 config is 100% matched with how black formats

[raven ridge]

sphinx is basically doxygen for Python. It's a documentation generation tool, which can read docstrings and use them to generate HTML docs.

[round lion]

...that sounds amazing

[round lion]

is there a reason you run mypy on an installed copy of the package rather than just pointing mypy to your src directory?

[swift imp]

I can't do static type checking

It's so way too immature and doing anything advanced as say descriptors and it breaks

[round lion]

the typing can be nice for readability and less maintenance on doc strings as going against the static typing won't raise errors, but yeah, forcing checks may be unreasonable unless you allow untyped functions, which then what's the point of running mypy against the codebase

[spark magnet]

don't add type hints if you don't check them. that's just asking for trouble.

[round lion]

yeah, I'd go all or none. Although for an existing codebase, it may be fine to integrate type hints gradually rather than all at once, then once it's all completed, forcing the check

[spark magnet]

do coverage first

[raven ridge]

mypy is a giant pain in the ass. I know running it on an installed copy fixes some problem. I don't remember what problem that is.

[round lion]

yeah, that's the easy one no one can complain about lol

[swift imp]

But that's the thing, we (at work) want to use them, but like then we get into this state where despite our best efforts we always have errors

[spark magnet]

it does seem to require some awkward casts.

[raven ridge]

There's always Any to shut it up

[round lion]

well, I have some stuff to research and test out now. thanks folks for all the input. no real python pros around me so I've been having to pick things up as I go and am tired of shooting myself in the foot lol

[swift imp]

But that like completely defeats the self documenting part

[spark magnet]

@round lion any time, we can help you shoot yourself in the foot 🙂

[raven ridge]

It documents "this is complicated, see the docstring", heh

[grave jolt]

-- I'm using C, a statically typed language, which provides much safety.
-- Oh, that's nice. Can you show an example?
-- Well,

void qsort( void *ptr, size_t count, size_t size,
            int (*comp)(const void *, const void *) );

[spark magnet]

"Any" == "void *" 🙂

[grave jolt]

yep, that's my point 🙃

[tidal marten]

You can typedef or define names for those types, but doesn't hide that void* is a bit dangerous 😂

[round lion]

you know, I was so happy I hadn't seen C code since college and you just had to bring it in here lol

[grave jolt]

"We don't need lambdas. Have you seen qsort_r?"

[grave jolt]

well, it's the same as

Int = Any
Str = Any
List = Any

[tidal marten]

At least C++'s lambda has type definition

[grave jolt]

If you're talking about specifying argument types in a lambda, then Pyright can also infer those if the target (where you pass the lambda) has a type.

But it refuses to infer them in other useful places (like functions under decorators) because "you can already specify the types in a standalone function"

[tidal marten]

So I just have to get better type checker or use libraries that have proper type hinting

Okay

[grave jolt]

or ||get a language with better gradual typing||

[tidal marten]

Yeah, but then I had to install their stupid toolchain (looking at you, Dart and Flutter)

[grave jolt]

typescript seems ok to me, but I haven't looked at dart&flutter

[gleaming rover]

TS is p good IMO

I quite like the syntax

apart from the angled brackets

[grave jolt]

Discriminated unions

i mean, sum types

jesus can we just settle on a single name

[tidal marten]

Typescript doesn't enforce runtime type checking tho

[grave jolt]

it does not

that's why it's gradual static typing and not just static typing

In fact, the whole point of static typing is that types are not checked at runtime.

But at compile time.

[tidal marten]

Well, that seems like duck typing with extra steps

[grave jolt]

runtime checking is the opposite of static typing

[spark magnet]

it's the same as C or Java

[grave jolt]

yeah, pretty similar

[spark magnet]

they don't do runtime checking either

[grave jolt]

to Java/C#, I guess

[spark magnet]

why not C?

[tidal marten]

C and Java won't allow you to call undefined methods tho

[grave jolt]

C doesn't have generics and subtyping

[grave jolt]

In C and Java you can do lots of nasty dynamically typed stuff.

[spark magnet]

sure, but it has a first step of checking types, and then runtime, no checking

[grave jolt]

well... all statically typed languages don't check types at runtime, right?

[grave jolt]

because

[spark magnet]

right.

[grave jolt]

(see void* and Object)

[tidal marten]

You have to explicitly want that. My problem with some Python libraries right now is that they does not do proper type hinting so my IDE is all red.

[grave jolt]

yeah

[tidal marten]

Yeah, all I want is at least proper type hinting lol

Wait, I thought in Java you can't cast an object to whatever you want, it still enforces type checking at runtime.

Or maybe that's where instanceof comes in

[grave jolt]

Well, sometimes you just can't typehint certain relations. Like in attrs or sqlalchemy.
mypy supports plugins, but other typecheckers don't, and it's a shame that they're not part of the typing standard. I would love "domain-specific type systems". Maybe I should just write my own typechecker like that... but that's just yet another standard

[grave jolt]

Yeah, but it will fail at runtime, which is bad: the main reason people use static typing is to provide some guarantees at compile time.

[tidal marten]

I'd rather my code fails that continuing with unexpected object with unexpected type to

[grave jolt]

More trivial example: you can't have a generic object with an unfilled type variable (i.e. polymorphic in some type) unless it's a function: https://github.com/microsoft/pyright/discussions/1380

[grave jolt]

Yes, sure.

But in compiled languages like C/C++/Rust, the type information is just not present at runtime.

[opaque raft]

Not quite an advanced question, but are you guys aware is sqlalchemy or any other pyhton orm supports getting model fields directly from indicated table so I do not have to define them?

[grave jolt]

Maybe this can help? https://pypi.org/project/sqlacodegen/

[opaque raft]

No, not quite

That's a generator

Does not work on the fly

[grave jolt]

Why do you want to generate them on the fly? Does the schema change at runtime?

[opaque raft]

I don't want to even generate them, I want each orm call to "determine" structure before it's executed

[raven ridge]

C doesn't have methods, but it does have functions - and it's both possible to call an undefined function at runtime, or to call a function with incorrect arguments at runtime.

[opaque raft]

I've been using PHPActiverecord in php project and am migrating it to python. That php library has that very handy property of determining schema and having columns pre-determined at each call

I'm just looking for same functionality in any of the python sql libraries

[raven ridge]

AFAIK, sqlalchemy does support that

[opaque raft]

Any idea what that would be called so I can hit the docs?

[grave jolt]

It just doesn't seem very useful to me. Your code will break unexpectedly, and your IDE will have no clue what you're doing.

[tidal marten]

You'd need to provide a function pointer with garbage address or address to wrong function, no?

[opaque raft]

My ide won;t break, I just use vim XD

[grave jolt]

You can have autocomplete and type inference in vim

[raven ridge]

No, sorry. I'm not a fan of ORMs.

[opaque raft]

I just want to define a class, indicate table name and viola....

[raven ridge]

Or fail to check the return of a dlsym call, or dynamically link against a library where the type of the symbol is different than the type you declared the symbol as when you compiled.

[opaque raft]

Me neither, however, sometimes it's handy to use new User() instead of writing sql.

[grave jolt]

@opaque raft Maybe https://sqlsoup.readthedocs.io/en/latest/tutorial.html?

well, it was last updated on 2016, so... maybe not

[opaque raft]

That's a good find, thanks mate. Some things don't need to change for a while, I don't mind.

[raven ridge]

the latter would be called "ABI incompatibility", and is a pretty common thing to need to worry about when making shared libraries

[boreal umbra]

@opaque raft this channel is for discussion about the language itself. If you're asking a question about how to accomplish a specific goal of yours, you might look into the topical channel that most closely relates to it.

[opaque raft]

Noted.

[boreal umbra]

why not

[grave jolt]

"There is no such thing as an object/relational mapping." (c) bob martin 🙂

seriously though, it's kinda true that a relational model does not exactly map to an object-oriented one

[raven ridge]

They're only very slightly easier than SQL for simple cases, and they're much worse than SQL for more complex cases

they frequently generate absolutely terrible queries with quadratic complexity that fail as soon as you deploy them to production and run them against real world data sets.

They make mistakes that a developer writing SQL would not make.

[grave jolt]

What are the benefits of ORMs? Not having to write complex queries?

(I haven't used one)

Do you think approaches like EdgeDB solve it better?

[raven ridge]

not having to write simple queries, really. Because from everything I've ever seen, developers still need to get involved and write the complex queries.

I'm not familiar with EdgeDB

[grave jolt]

well, it's basically a relational database that's built on top of postgres, but provides a sorta higher-level language than SQL

[raven ridge]

The advantage of an ORM is that it will get the data you ask for, and you don't have to think about how it's doing it, even if that takes joins across tables without suitable keys. The disadvantage is... exactly the same thing.

[boreal umbra]

I thought the point of them was that once you've obtained a row from a database, that row is represented as an object that you can have defined methods for

but you're telling me that the main point is that it figures out what query will obtain the information one wants?

[raven ridge]

They're much more than just dataclasses for tables.

They usually do things like lazily fetch data for that object, so you can pretend that you've got a local object when it actually fetches stuff from the DB behind the scenes as needed.

[boreal umbra]

that's been my experience with them. so what does that have to do with how they're obtained via query?

(I was mostly done with that comment when you said the next thing and I wasn't going to let my effort go to waste.)

what if I just want dataclasses for tables?

[lethal thicket]

in bs4 if i have a header object inside of a class in the html code how do i retrieve that header

[grave jolt]

This channel is for discussing the Python language itself. You should see #❓｜how-to-get-help

[raven ridge]

https://docs.sqlalchemy.org/en/13/orm/tutorial.html#building-a-many-to-many-relationship shows this statement: py wendy.posts.filter(BlogPost.keywords.any(keyword='firstpost')).all() Running this SQL: ```sql
SELECT posts.id AS posts_id,
posts.user_id AS posts_user_id,
posts.headline AS posts_headline,
posts.body AS posts_body
FROM posts
WHERE ? = posts.user_id AND (EXISTS (SELECT 1
FROM post_keywords, keywords
WHERE posts.id = post_keywords.post_id
AND keywords.id = post_keywords.keyword_id
AND keywords.keyword = ?))

granted that's not incredibly complex, but it does show joining across multiple tables. There's a lot of join type stuff in there.

as far as just wanting dataclasses for tables - I'd just make my own dataclasses, and load data into them myself after reading it from the database.

[grave jolt]

...or even automate that somehow, e.g. with dataclass_factory and a mapping-Row (i.e. a row that presents a key-value-pair interface)

[raven ridge]

yeah, exactly. But if you're feeling lazy, a dict row factory, and just passing the row dict to your dataclass constructor as **kwargs, does the trick.

[grave jolt]

yep

but then... why would you want that?

That's just revealing a whole bunch of implementation details to the users of those classes

maybe it's kinda useful when you're directly fetching data, though.

e.g. if you make a change to a table row, you don't need to fix every single place

[raven ridge]

I mean, I wouldn't expose them to users of your library, or whatever - I'd keep those classes as implementation details. And I'd only use classes for it instead of just POD dicts if I needed to have methods that compute non-trivial derived properties, or something.

but yeah, I mostly agree - I don't usually create classes that represent tables.

[tidal marten]

I have multiple instances where I knew which queries that I want to run, but didn't know which ORM operator to use.

ORM is mostly a hinderance when you have complex queries. It also locks you into a particular ORM or MVC ecosystem.

[raven ridge]

Yeah. And replaces a well documented, relatively portable language with an ad hoc, less well documented DSL tied to a particular programming language

[sacred tinsel]

Our setup is similar, we run flake8 with some plugins (mostly import order, docstrings, type annotation coverage) and a unittest testing suite. We report test coverage but don't enforce 100%. I like using flake8-annotations to enforce ann coverage, even if there's no type checking, I think the annotations still have massive value in providing a standardised way to document types, even if you don't run a type checker ~ the docstring can then focus only on what the annotations cannot express, and there's no debate on which param documentation style to use. For new projects or "my" projects where no one else contributes, I'm starting to use mypy, but I probably wouldn't be able to introduce it to larger existing projects because the devs either don't have the capacity to fix hundreds of issues, or would rather spend their efforts elsewhere. PyCharm does type check for me to some extent that is useful, plus it can autocomplete a lot more if you provide the annotations, which is also super useful, although I find that it won't catch nearly as many issues as mypy & alternatives would.

[grave jolt]

Typecheckers are extremely useful IMO when it comes to more advanced things like generics.

Like, quiz: can you do

T = TypeVar("T", covariant=True)

class Observer(Generic[T]):
    def update(self, new_value: T) -> None:
        raise NotImplementedError
```?

Or, can I pass list[int] to a function that expects list[Union[str, int]]?

I understand that non-libraries don't create generics often, though.

[grave jolt]

Maybe the issue you have is that an expensive operation looks very cheap?..

Like putting lots of I/O in a dynamic property lookup

[peak spoke]

To me, even just using simple typing (skipping the more complex types by using a more generic type and documenting the param in more detail in the docstring) seems to be great value from what it enables on editors and checkers. I've been mostly relying on pycharm's builtin checker which works for me without worrying about it too much

[sacred tinsel]

I don't know, can you? I'd be more worried about whether you can pass [1, "1"]

now I realized I don't even know what the union expresses there

[grave jolt]

list[int] is a list, where each element is int: [1, 2, 3, 4, 5, 6]
list[Union[int, str]] is a list, where each element is Union[int, str]: [1, "abc", 2]

[sacred tinsel]

right, so Union[list[int], list[str]] would be the way to express either list[int] or list[str], but not a mixture

[grave jolt]

Yes.

[sacred tinsel]

is there a way to express that it has to be a mixture? i.e. at least one str and at least one int

[grave jolt]

No, that's not really possible

but not sure when it would be useful

[sacred tinsel]

so you would need a custom type

e.g. a class that would enforce the constraint upon instantiation

[grave jolt]

Well, yes, you could have a custom type like

class NonEmptyList(Generic[T]):
    def __init__(self, first: T, rest: list[T]):
        self.first = first
        self.rest = rest

    def __getitem__(self, index: int) -> T:
        if index == 0:
            return self.first
        elif index == -len(self):
            return self.first
        else:
            return self.rest[index-1]

    def __len__(self):
        return len(self.rest) + 1

    def __iter__(self):
        yield self.first
        yield from self.rest

...to encode a non-empty list. In a similar way, you could do

class MixedList(Generic[T, U]):
    def __init__(self, first: T, second: U, rest: list[Union[T, U]]):
        self.first = first
        self.second = second
        self.rest = rest

[grave jolt]

The answer here is no, because this function is absolutely valid:

def foo(things: list[Union[str, int]]):
    things.append(42)
    things.append("bar")

In nerd speak it means that list it invariant in its type parameter: https://mypy.readthedocs.io/en/stable/generics.html#variance-of-generic-types

[sacred tinsel]

right, but that only applies if you mutate the param right

[grave jolt]

Well, if you're not mutating the parameter, you should use Iterable, Sequence or Collection depending on what you need.

[sacred tinsel]

so does this annotation lie?

def add_string(ints: list[int]):
    ints.append("1")

I guess yes, although I never really looked at it that way

I've never considered the annotation to also be a promise of what the type will be after the function exits

but yea, this makes sense

[grave jolt]

yup

You should post this in #data-science-and-ml, not here. This channel is for discussing the Python language itself.

[bright lantern]

Oh sorry. My bad

I will post in data science

[gleaming rover]

you need dependent types for that

[sacred tinsel]

okay, so how do you deal with a function that takes a list of ints, sums them, and appends the sum as a string:

def add_sum(ints: list[int]):
    ints.append(str(sum(ints)))

[gleaming rover]

because aside from the types of the contents you’re expressing a constraint on value (length)

[sacred tinsel]

yea, the fact that there's a constraint on the length also occurred to me

[grave jolt]

Well, there's no way for you to describe the type of a mutation... so you can't really do that, besides list[Any] and a docstring

[gleaming rover]

what do you want to express?

[grave jolt]

@sacred tinsel @gleaming rover Well, you can do it like this. But it restricts to the first 2 elements.

@sacred tinsel Subtyping is generally bit tricky, because any given value belongs to potentially infinitely many types. And mutation makes it all even worse.

[sacred tinsel]

I need to express that the function can only take list[int] so that I can sum them, but I also append a string, so then the list no longer satisfies list[int]

however it cannot take list[Union[str, int]], because then it will fail

[gleaming rover]

no

[flat gazelle]

the python type system cannot express this

[gleaming rover]

the parameter constraint applies only before the function is entered

[grave jolt]

the best you can do is return a new list

[sacred tinsel]

so in my mind annotating the param as list[int] would be the most correct, because it expresses what the function needs to receive

[flat gazelle]

the core issue here it that a single variable may not change its type

[sacred tinsel]

(ignoring that you can sum other types too)

[flat gazelle]

so you have immutability here, yet in practice the type of a variable is mutable

I am not aware of any language that can deal with this correctly

[grave jolt]

I guess you could do it with linear types, i.e. ensuring that it's not used anywhere else:

def add_sum(ints: consumes[list[int]]):
    # here ints is list[int]
    ints.append(str(sum(ints)))
    # here inst is list[int | str]

[unkempt rock]

I wonder if its possible using *args + paramspec

[spark magnet]

@sacred tinsel the problem might be appending a string to a list of ints in the first place. That seems odd.

[grave jolt]

[sacred tinsel]

sure, it's a theoretical scenario

[grave jolt]

If you want to get some theoretical scenarios Python's type annotations can't express, I can open up my type annotation frustration bag 🙂

[sacred tinsel]

feel free to give a sample of your bag, but you may not be able to get much discussion out of me

I did come across a more practical problem recently

is this the correct way to annotate that the function receives MyEnum, the class?

I think so, but PyCharm doesn't like it

line 14 should be creating a member just like line 10 does

[spark magnet]

why t.Type[MyEnum]? Can't you annotate it as MyEnum?

[sacred tinsel]

I think that means it takes a member

it takes a class (MyEnum), and creates a member MyEnum.A

[spark magnet]

oh, sorry, you want to take the class

in that case, you are trying to say that this must always take MyEnum as the argument?

[sacred tinsel]

in my scenario it would be a union of two enum classes

so Union[t.Type[MyEnumA], t.Type[MyEnumB]] I think

but here in this example it only takes one, trying to isolate the issue

the code works, I'm just not sure whether it's not annotated correctly or maybe just a PyCharm issue

[flat gazelle]

mypy doesn't complain, so it may be a pycharm issue

[spark magnet]

i think sometimes it is not worth trying to get the type system to understand.

[flat gazelle]

ye, just go with Any and write a docstring

[sacred tinsel]

mm yea

[spark magnet]

or leave off Any completely.

somehow with git we forget about "decentralized" and with Python type hints we forget about "gradual"

[peak spoke]

It is good to be explicit that you didn't forget the annotation there but wanted any, even though the behaviour is the same

[weary garden]

All my C++ numerical (arbitrary precision arithmetic) library tests pass! \o/ Now I can start work on my implementation of Python!

[weary garden]

https://github.com/i42output/neonumeric

[rose stream]

Does anyone know how to get autocompletion using getattr?

[spark magnet]

we're looking forward to it.

[raven ridge]

Can you elaborate?

[rose stream]

class Vcc:
    
    def __init__(self, vcc:VccType):
        self.vcc = vcc

    def __getattr__(self, name) -> str:
        
        if (valid_attr := self.vcc.get(name)):
            return str(valid_attr)
        
        raise AttributeError(f'{name} is not a valid attribute.')

[spark magnet]

@rose stream implementing __dir__ might help?

[rose stream]

    def __dir__(self):
        return self.vcc.keys()

[silk pawn]

also shouldn't _ only be used for things that are going to be garbage collected next time it runs anyway

i feel like a better name for the variable would be anything other than _, since you're accessing an attribute of it later

[spark magnet]

not that it matters to your point, but Python reclaims objects a little differently than waiting for gc to run.

[silk pawn]

oh

how does it reclaim objects then?

i thought it reclaimed objects when ref count dropped to 0 and garbage collection was run on the object

[spark magnet]

if the ref count is 0, the object is reclaimed immediately.

[silk pawn]

oh

[spark magnet]

there's a gc that runs to collect circular references

[silk pawn]

how does the gc know when to collect circular references

[grave jolt]

GC is the circular references detection part

(mostly)

[spark magnet]

https://docs.python.org/3/library/gc.html

[silk pawn]

ok thanks

[grave jolt]

so IIUC, if you don't have any circular references, things will just die when they go out of their scope

[spark magnet]

names release references to objects when the names go out of scope. objects are reclaimed when they have no more references.

https://nedbatchelder.com/text/names1_pix/024.png

[grave jolt]

right

[raven ridge]

https://devguide.python.org/garbage_collector/ explains it in excruciating detail, if you're very curious

[feral cedar]

there is also a cool talk

<
https://www.youtube.com/watch?v=CLW5Lyc1FN8>

[weary garden]

can anyone recommend a good python book good/advanced enough to aid me in what I am doing?

I have Pocket Reference and Cookbook already

what I really need is a reference book

[grave jolt]

@weary garden Well, the most 'reference' reference is
https://docs.python.org/3/reference/index.html for the language
and https://docs.python.org/3/library/index.html for the library
if you want to see how a particular feature (e.g. pattern matching) is specified, you can read the appropriate PEP or ask about something unclear here.

[weary garden]

PEP?

[grave jolt]

are you trying to implement Python but never heard of PEPs? 🙂

[weary garden]

yes

[grave jolt]

they are basically RFCs

https://www.python.org/dev/peps/

[weary garden]

I see.

so, it begins...

opens wine

I assume the Python language itself does not have any copyright related encumbrances?

[grave jolt]

Well, there already are multiple implementations, so... You might want to ask in a mailing list.

[peak spoke]

considering the amount of implementations that popped up over its lifetime I imagine it's not a very big hurdle even if there are things to follow

[weary garden]

\o/

[limpid forum]

wait, neos? are you that person who was bragging about their "cross-platform all-language compiler" in some python mailing list?

if yes, I'm ready for the proof of concept ❤️ , the discussion there was just a lot of words from a lot of people

[spark magnet]

I think many people are eager to see something runnable.

[grave jolt]

well, in some languages being able to compile is more exciting than running 🙂

but C++ is not one of them

[spark magnet]

A runnable compiler counts as something runnable in this case 🙂

[grave jolt]

https://github.com/susisu/typefuck

I wouldn't be surprised if someone implemented Python in C++'s templates.

so it's not so clear cut

[weary garden]

what does it mean to run a python program that just consists of an expression? in interactive mode the expression is evaluated and printed but what it it is a program?

[grave jolt]

I don't think I understand your question

[weary garden]

foo.py contains

123

and nothing else

python foo.py

[grave jolt]

The expression gets evaluated, and its result is discarded.

e.g. print("hello, world!") is an expression

it's evaluated, and its result -- None -- is discarded

(much like in C++)

[weary garden]

if it is just an integer it doesn't try and do anything with program return code?

[spark magnet]

no

[grave jolt]

no

[weary garden]

good

carry on.

[spark magnet]

you can try running these programs to see what they do

[weary garden]

I am doing

[spark magnet]

did it set the status code?

[grave jolt]

Can't run a program without a compiler 🙂

[weary garden]

I didn't write a script to check it

[peak spoke]

Can run the reference implementation though

[weary garden]

I am basically asking about the differences between interactive mode and compiling/running a program

[grave jolt]

A standalone file is like a body of a main function

[peak spoke]

I was actually wondering about this when helping someone, https://docs.python.org/3/reference/simple_stmts.html#expression-statements is the documented grammar here correct? As I understood it the expression statement would for example be a single expression on a line but a starred expression is not valid in that context

[weary garden]

can you define local functions (not lambdas)?

I suppose I could tias

[spark magnet]

what starred expression do you mean?

[peak spoke]

*name for example

[spark magnet]

right, that's not an expression

[weary garden]

so it has local functions. surprised. +1 for Python

[spark magnet]

why does that surprise you?

[grave jolt]

yes, a lambda is not special in that regard.

C++ is not like that 🙂

[weary garden]

because C++, C, Java and lots of other languages don't

the only way to get a local function in C++ is to use a lambda

or a local struct

no I meant local function, not lambda, @grave jolt

[grave jolt]

In Python, lambdas are only different in that they are an expression and that they don't have a name.

Local functions are closures as well.

[weary garden]

something very strange has happened

[spark magnet]

Python is not like a lot of other languages.

[weary garden]

very strange indeed. I must be getting old.

I thought I would find whitespace-based block scoping egregiously irritating but I am actually finding it "meh".

[peak spoke]

What is it then? I'm not exactly sure how to interpret the grammar under https://docs.python.org/3/reference/expressions.html#expression-lists but to me it looks like it'd fall under it

Within python I find it quite clear with just indentation around, makes for less noise and the braces or other means of denoting blocks aren't necessary with how the language is structured

[grave jolt]

Sass is an example of significant whitespace implemented not so well. In particular, it has a very hacky way of 'inserting braces' that makes it impossible to split a line into multiple lines AFAICT.

[spark magnet]

you have chosen a strange path for learning python, but it's similar to the well-worn path. many people think significant indentation will be annoying, then they realize they indent anyway, and it's good.

[grave jolt]

But it means that I can't make a doughnut-shaped doughnut renderer

[spark magnet]

true, the obfuscated C contest has much more lexical innovation

[weary garden]

are there any keywords that can appear within function scope that cannot appear at outermost scope?

[peak spoke]

a few

[spark magnet]

"global". have you considered learning the language? 🙂

[weary garden]

that is what I am doing

[grave jolt]

nonlocal and global, that's it, I think.

[cedar ice]

Return?

[grave jolt]

right

[spark magnet]

yup

[grave jolt]

yield, yield from, await

[weary garden]

thanks it is sufficient for me to know there is at least one thanks

[grave jolt]

async for, async with

[weary garden]

I don't suppose there is a language grammar anywhere I can refer to so I don't have to ask so many questions?

[grave jolt]

there is

[grave jolt]

In the reference, there's a Grammar section

https://docs.python.org/3/reference/grammar.html

[weary garden]

great!

I don't see significant whitespace in the grammar; I guess you woudln't if the grammar is context free.

[grave jolt]

well, it doesn't reveal all the parsing details

[weary garden]

however my neos schema file has to

I am guess that eval is when you pass a Python expression as a string to some kind of eval function (reflection?) but what is func_type and fstring?

[gleaming rover]

@grave jolt having thought about it

I believe there is no static type system in which the type of a variable can change throughout a program

does that sound true to you

[grave jolt]

I suppose that it's a function signature, including typehints.

[grave jolt]

i haven't encountered one

[flat gazelle]

Rust's move could be considered this

[weary garden]

@gleaming rover have you considered variants? (discriminated unions)

[grave jolt]

that's not the same as changing a type

The type is the same

[gleaming rover]

^

[gleaming rover]

could you elaborate

[grave jolt]

e.g. a fixed-size integer is just a discriminated union of 0 | 1 | 2 | ...

[weary garden]

the type of the object held by the variant can change dynamically at runtime

[grave jolt]

The type is the same.

[gleaming rover]

justice for -1

😡

[grave jolt]

Or, well... define what you mean by 'type'

[weary garden]

the type of the variant is the same

but the type of the contained object can change

from a list of types to select from

in C++ at least

std::variant

[flat gazelle]

essentially

let a = Obj::new(); // a owns the obj
let b = a; // a moves to b
let c = a; // error, a has moved 
``` a changes from not moved to moved during the program

[gleaming rover]

this looks like it’s related to linear types

[flat gazelle]

somewhat

they can be used to describe similar semantics

but rust's system isn't exactly linear types

[gleaming rover]

actually

[grave jolt]

@weary garden In my understanding, a type is a set of allowed operations on a value. The set of allowed operations doesn't change if the value of the variant changes.

[gleaming rover]

not linear types

[cedar ice]

It's affine types

[gleaming rover]

affine types

ye

I always find it hard to remember

[weary garden]

with a variant you can destroy the contained object of type A and create a new object of type B, A and B are from the list of types the variant is defined to support

[gleaming rover]

which is which

[cedar ice]

Variants are sum types, they aren't allowing type changes

[gleaming rover]

but I believe from a type theory point of view

the type does not change?

[cedar ice]

Yep, it does not

Static affine types

[gleaming rover]

rather, the type system imposes constraints on usage

[flat gazelle]

ye, IG the possible set of values a may hold remains the same

or well, to an extent, a changes from type Obj to the Void type, since there is no value a can be once that happens

[gleaming rover]

let me clarify the question

[weary garden]

no new type is created or destroyed at runtime (types are only defined at compile time) however the variant allows you to store objects of different types within the variant without changing the type of the variant itself.

[grave jolt]

define 'type'

I think you use that word in two different meanings

[weary garden]

std::variant<foo, bar> v; v can hold objects of type foo or bar

[gleaming rover]

is there any type system which allows widening of the type of a variable, i.e. a typesafe enlargement of the set of possible values that variable may hold, at some point post-definition?

and to me that goes against the notion of a type system...?

[flat gazelle]

I could imagine ada doing this for range types

but I do not know for sure

ada, not agda, mb

[gleaming rover]

nobody disputes this

[spark magnet]

so long as we are clear what language we're talking about.

[gleaming rover]

but any value you wish to assign to that variable

is one that you COULD have assigned to it at any earlier point

[cedar ice]

That's not something that holds two types, it's a type that is the union of two types. They're called sum types in type theory

[gleaming rover]

i.e. the set of values that may be assigned to it never changes

[grave jolt]

Any conditions you will add further can only narrow a type. For example, in TypeScript/Python, or something bizarre like ATS

but maybe...

idk

[weary garden]

@cedar ice It can only hold an object of one of those types at any time during runtime and you can change which that is at runtime

[flat gazelle]

that's a sum type

[cedar ice]

Yes, you don't have to teach me C++

It's a sum type. Sum types are an example algebraic data type, just like product types (aka tuples)

[gleaming rover]

did someone say QUOTIENT TYPE?

🥴

[cedar ice]

Where the possible values for variable are either unions of two type sets (sum types) or Cartesian products (product types)

[gleaming rover]

that was m first question

like if we have sum types and product types

what about - and /

it was hard to even Google

[flat gazelle]

typescript can subtract types

[weary garden]

values are objects which aren't types

[gleaming rover]

what is a kind then

[spark magnet]

does everyone but me know what language we are talking about right now?

[grave jolt]

I have lost track already

[flat gazelle]

we are talking about type theory

[gleaming rover]

a way to say “you can only use this variable exactly once”

[weary garden]

I think we are talking about Esperanto

[gleaming rover]

or “at most once”

[cedar ice]

They're just move semantics. Linear types must have only one owner and must be used somewhere, while affine are max one, but also 0

[spark magnet]

this statement definitely needs qualification

[gleaming rover]

otherwise it’s a type error

[oblique crystal]

still python no? I could not get follow the full convo 🙂

[flat gazelle]

type division would probably be setting one part of a product to a value, which sounds like something dependently typed languages can do

[weary garden]

this is "advanced discussion" so I assume advanced stuff like this is on topic

[cedar ice]

The value of a variable Y of type A can take a form of any of the values of the possible values for type A. The value of a sum type (of types A and B) variable X is one of the possible values of two separate types A and B.

[spark magnet]

it is, but some words are true or false depending on what language we are describing

[cedar ice]

In contrast, product type must take on a value of both type A and B, eg a tuple with both types A and B

[cedar ice]

This is generic type theory that applies to all languages, not specific to any languages

[swift imp]

This sounds like non sense

[spark magnet]

your statements have been, yes. others have not been.

[cedar ice]

Fair enough

[grave jolt]

If I'm confused, I use this heuristic:

if (I see Greek words):
    it's something about category theory
elif (I see Latin words):
    it's either an exorcism or something about abstract algebra
elif (I see English words):
    it's probably type theory
elif (I see Python):
    I'm in #esoteric-python

[flat gazelle]

not to be confused with what python calls types, which are not type theory types (unless you really contrive)

[weary garden]

in general a value or object is an instance of a type; this what I said.

[spark magnet]

this could be debated

[cedar ice]

x=type(3)

[grave jolt]

if ```
a/b = c <=> a = c * b

(int, str) / str = int

[flat gazelle]

yup

[gleaming rover]

a quotient type

[grave jolt]

time to find a remainder type

[weary garden]

x=type(3) is an object of type(type(3))

I assume

[swift imp]

I feel like I'm in haskell

[grave jolt]

yeah, let's perhaps move to OT

[cedar ice]

Haskell gets sooooo much worse

[flat gazelle]

not wrong

[grave jolt]

!ot

[fallen slate]

Off-topic channels

There are three off-topic channels:
• #ot0-psvm’s-eternal-disapproval
• #ot1-perplexing-regexing
• #ot2-never-nester’s-nightmare

Their names change randomly every 24 hours, but you can always find them under the OFF-TOPIC/GENERAL category in the channel list.

Please read our off-topic etiquette before participating in conversations.

[gleaming rover]

means a type that comes with an equals method (basically)

[gleaming rover]

fortunately

not necessarily in Python 😉

[weary garden]

I don't see why this discussion is off topic for advance discussion

[gleaming rover]

🐍

[flat gazelle]

we are getting more to general type theory than python

[gleaming rover]

because this channel is for discussion of the language

[swift imp]

Which offtopic are you moving to

[grave jolt]

#internals-and-peps is

Discussion on the use cases, implementation and future of the Python programming language including PEPs, advanced language concepts, new releases, the standard library, and the overall design of the language.

ot1

[cedar ice]

Type theory relevant to python eg talking about type(3) being a type object should be on topic yeah?

[spark magnet]

yes

(imho)

[weary garden]

when discussing Python implementations the discussion of other languages and language-agnostic concepts becomes necessary.

[gleaming rover]

the ORIGINAL reason I brought it up

was a question yesterday about the appropriate type annotation for a certain function

however, we have digressed

[grave jolt]

Yeah, but not affine types

and such stuff that doesn't really apply to python tbh

[weary garden]

😄

[grave jolt]

dd

if you allow tabs, you open up pandora's box

[weary garden]

I assume "TabError: inconsistent use of tabs and spaces in indentation" means you can't mix tabs and spaces in the same indentation?

[grave jolt]

Yep. Which means you can't align stuff.

IIUC

And the standard style is 4 spaces.

actually, I'm not sure if the language allows it lol

[weary garden]

unfortantely I cannot not allow tabs if CPython allows tabs

[grave jolt]

right

[raven ridge]

""" is not a multi-line comment, it's the start of a string literal. And you can also have '''

[flat gazelle]

You can align stuff,

if 5:
    a=[
1,
2,4,4,5
,67]``` is valid python IIRC

[weary garden]

so people use string literals as comments?

[grave jolt]

Docstrings aren't really usual comments.

They can be (and are) inspected.

!e

def fizz():
    """buzz"""

print(fizz.__doc__)

[fallen slate]

@grave jolt :white_check_mark: Your eval job has completed with return code 0.

buzz

[raven ridge]

Sometimes people use them for comments as well, but in particular contexts they're introspectable.

[weary garden]

"Since Python will ignore string literals that are not assigned to a variable, you can add a multiline string (triple quotes) in your code, and place your comment inside it:"

at this point I would usally swear like a trouper

[raven ridge]

!e ```py
def fizz():
r'buzz'
print(fizz.doc)

[fallen slate]

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

buzz

[grave jolt]

that as well^

[raven ridge]

That works with other types of string literals, too.

[grave jolt]

Well, you technically can do that for general multiline comments, but mostly nobody does that, at least from what I've seen.
https://www.python.org/dev/peps/pep-0008/#comments from what I can tell PEP 8 recommends #

[raven ridge]

You can do the same in C++. You could use

"Foo"
"Bar";

As a comment. It's weird, but works.

[raven ridge]

It's used occasionally for quickly commenting out a block of code. If you never use ''' for anything else, you can use it to disable a chunk of code regardless of what's inside

Like #if 0 / #endif in C

[weary garden]

@grave jolt my quote was from the W3C website

[pliant tusk]

oh python string literals also implicitly concatenate. ```py

x = 'a' 'b'
print(x)
ab

[grave jolt]

you mean, w3schools 😄

[raven ridge]

It says you can do that. It's correct. @grave jolt says that no one really does that. That's also correct.

It's possible, but not idiomatic.

[grave jolt]

it's also correct that w3schools is not a very good place to become familiar with the idiomatic use of the language.

and they're not affiliated with w3c

[weary garden]

I assume the single double quote deal is so you can easily embed single quotes in a double quoted string and vice-versa?

[grave jolt]

Triple quotes allow for multiline strings

[weary garden]

no I mean print(" ' ") and print(' " ')

[grave jolt]

ah

[raven ridge]

Yep. They're semantically equivalent. The only difference is what you need to escape in it

[grave jolt]

yes

[weary garden]

this will make for an interesting parse rule

[raven ridge]

There's lots of types of strings. https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals

[grave jolt]

!e FR"oui"

[fallen slate]

@grave jolt :warning: Your eval job has completed with return code 0.

[No output]

[raven ridge]

Heh

Also, like C, adjacent string literals are implicitly concatenated at compile time.

!e ```py
print(
"A"
"""B"""
'C'
)

[fallen slate]

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

ABC

[grave jolt]

tbh this is a bit unexpected and unobvious, and it has bitten me

!e

xs = [
  "foo",
  "bar",
  "baz"
  "bonk",
  "fizz",
  "buzz",
]
print(xs)```

[fallen slate]

@grave jolt :white_check_mark: Your eval job has completed with return code 0.

['foo', 'bar', 'bazbonk', 'fizz', 'buzz']

[raven ridge]

Well, don't do that 😂

[grave jolt]

I meant that I missed a comma

[raven ridge]

I meant that you shouldn't

[grave jolt]

Well, I shouldn't make mistakes -- that's pretty solid advice 🙂

[raven ridge]

I think I've done that... More than never, but not often.

I've definitely used it intentionally more than it has ever bitten me accidentally

[grave jolt]

!e
this one actually raises a warning:

xs = [
  ("foo", "bar"),
  ("baz", "bonk")
  ("fizz", "buzz"),
]

[fallen slate]

@grave jolt :x: Your eval job has completed with return code 1.

001 | <string>:3: SyntaxWarning: 'tuple' object is not callable; perhaps you missed a comma?
002 | Traceback (most recent call last):
003 |   File "<string>", line 3, in <module>
004 | TypeError: 'tuple' object is not callable

[grave jolt]

I wish it said you cannot call a tuple so that it's not as cryptic to beginners, but no big deal

[weary garden]

😄

[grave jolt]

why can't you just specify it in EBNF-like language?

[weary garden]

because this grammar isn't context free

it also encodes semantics

not just syntax

[grave jolt]

You can make a lexer that turns indents and dedents into tokens.

[grave jolt]

Like lark did:
https://github.com/lark-parser/lark/blob/master/examples/advanced/python_parser.py
https://github.com/lark-parser/lark/blob/master/examples/advanced/python3.lark

[weary garden]

"string_literal" is an artefact of the Python schema definition however "string.utf8" is a neos semantic concept

[raven ridge]

What are the semantics? A Python str is Unicode, but not UTF-8.

[grave jolt]

Well, it doesn't specify whether it's utf8

but operations like indexing will be inefficient

[weary garden]

no

this is the encoding of the literal text

in-source

[raven ridge]

Ah. That's configurable in Python.

[weary garden]

remember neos is a universal compiler than can compile any programming language; I am hoping there will be no Python specific semantic concepts required but it won't be a big deal if there are

[grave jolt]

The schema just seems so verbose compared to something like

single_quote_string: "'" (ESCAPE | /[^'\n]/)* "'"
double_quote_string: '"' (ESCAPE | /[^"\n]/)* '"'
triple_single_quote_string: "'''" (ESCAPE | /[^'\n]/ | /'(?!'')/)* "'''"
triple_double_quote_string: '"""' (ESCAPE | /[^"\n]/ | /'(?!"")/)* '"""'
ESCAPE: "\\n" -> LF
      | "\\r" -> CR
      | "\\t" -> TAB
      | "\\" /./ -> JUST

you're basically hand-writing a parser.

[weary garden]

as I said earlier I am describing both syntax and semantics so traditional CFGs dont' cut it.

[grave jolt]

why are you mixing syntax and semantics?

what context is needed to parse Python?

lark does it in a context-tree grammar just fine.

CPython as well AFAICT

[weary garden]

just a single schema file; as I said neos is a universal compiler not a Python compiler

[grave jolt]

But why can't the universal compiler work with a concise syntax description that looks very much like the full grammar reference?

[raven ridge]

https://www.python.org/dev/peps/pep-0263/#defining-the-encoding - the encoding for a Python file itself is specified as a magic comment on the first two lines of the file. The contents of a string literal in the file aren't necessarily UTF-8, they could be any other encoding known to Python (including encodings that have been dynamically registered by other files that have already been imported)

[grave jolt]

(minus the custom lexing part where you describe dedents and indents)

[weary garden]

because syntax is 50% of the problem: the other 50% is semantics and I wish to describe the syntax and semantics of any programming language with a single schema file in Relaxed JSON format.

[limpid forum]

sub-part of "universal compiler" would be a "python compiler"
if you give it only python code, it will work like a python compiler, won't it?

[grave jolt]

Right, but so far it's just half of the description of string literal syntax, and it's already 1/4 as long as
https://github.com/lark-parser/lark/blob/master/examples/advanced/python3.lark
(okay, it uses some shortcuts, but still -- much less verbose)

[weary garden]

@limpid forum https://neos.dev for the details

[limpid forum]

yes, I know this. I read your mailing list discussion bragging about it as well
and it had the same question - "universal compiler" doesn't change the fact that part of it has to compile python, so it's also a "python compiler"

any "universal compiler" would need to be also a "python compiler"
not the other way around

[weary garden]

not really, the schema file is an INPUT to the compiler; source code meta data if you will.

the compiler doesn't give a hoot about what language it is compiling

[limpid forum]

we're talking about two different things
I'm just arguing over semantics of "it's an universal compiler, not a python compiler", not how you define python in it

[raven ridge]

if it's able to compile Python code, it's a Python compiler, regardless of what else it can do.

[weary garden]

I guess we are arguing semantics then 🙂

I prefer to call it a universal compiler because the set of languages it can compile isn't closed.

[limpid forum]

I just don't like how you deflect any comment by saying "it's a universal compiler, not a python compiler"
because if something deals with shapes and you feed it a triangle, you deal with triangle. a sub-part of your something deals with triangles. and you're just repeating "no, it's not dealing with triangles, it's dealing with shapes"

[weary garden]

OK, I will agree with you that a universal compiler than can compile python programs is also a python compiler. 🙂

[limpid forum]

and as I said several times now, it's not the first time I've seen you doing this

[sacred yew]

@weary garden how are you handling incompatabilities between different minor versions of python?

i see your schema file just says "python 3.x"

[weary garden]

@sacred yew Not really thought about that; you can have different schema files I suppose

[sacred yew]

@weary garden what about the stdlilb

same for the other languages

[weary garden]

case-by-case basis. Python buitlins will require some work otherwise it is just importing Python code and/or using libffi for C libs

[raven ridge]

right, but the Python code in the standard library changes between versions.

[weary garden]

the schema file will specify the location of the stdlib files for that version then. it isn't something I am majorly worrying about at the moment.

[pliant tusk]

also im curious what your plan is for passing neos esq objects to C libs. they still expect PyObjects

[grave jolt]

I assume it will be incompatible with existing C extensions?

[weary garden]

I do not intend to have any compatibility with CPython as such so it I will use whatever approach other non-CPython implemntors use

[raven ridge]

some build a compatibility layer with CPython extensions - like pypy.

the others... don't get used much, heh.

[weary garden]

I will look at how I am going to manage stdlibs and such once I have got the core language working

Python will be the first language that I am implementing using neos as Python seems to be the most popular language in use at the moment. Got to keep the hipsters happy. 😄

[sacred yew]

isn't half the stdlib implemeneted in C

including the builtins

[raven ridge]

yeah.

[flat gazelle]

You have pypy's stdlib, which is in python

[raven ridge]

though many of them also have reference implementations in Python.

[sacred yew]

pypy's stdlib uses its own internal modules though

[weary garden]

a lot of the builtins will be implemented in python that call into the language agnostic neos stdlib

[sacred yew]

i see

  
import time

from pypy.interpreter.error import oefmt
from pypy.interpreter.typedef import TypeDef
from pypy.interpreter.gateway import interp2app, unwrap_spec
from pypy.interpreter.baseobjspace import W_Root
from rpython.rlib.rarithmetic import r_uint, intmask, widen
from rpython.rlib import rbigint, rrandom, rstring

as the imports in _random for instance

[raven ridge]

a lack of support for CPython C extensions will be a major roadblock standing in the way of your compiler/interpreter getting any adoption outside of special cases, though.

[weary garden]

maybe but the main reason I am doing this is for neoGFX.

[raven ridge]

Libraries like numpy are pretty important to many people; inability to support them will stop any widespread adoption.

as long as widespread adoption isn't a goal, that's fine. 🤷

[weary garden]

my goal is to let people use their favourite language as a scrpting language for neoGFX: think Unreal Engine with Python scripts for creating games.

[sacred yew]

shudders c++ as a scripting lang?

[weary garden]

I don't plan on creating a C++ implementation given that C++ is the native neos language and C++ is huge.

[raven ridge]

I'm not certain Python is smaller than C++. Roughly the same size, I think.

[weary garden]

well the Python grammar doesn't look very big

stdlibs maybe

[grave jolt]

I would doubt that...

[sacred yew]

*c as a scripting lang

[grave jolt]

C++ has a very complex type system

and now concepts

it's like the meme of a complex language

[raven ridge]

true, I'm not caught up on C++20. I'm familiar with 11/17

[weary garden]

neos will require C++20

[sacred yew]

on a related note, are there build instructions for neos anywhere?

[raven ridge]

but the grammar isn't the biggest part of the language. Python has tons of complexity buried beneath the grammar.

[weary garden]

not yet. as I am working on this project myself I don't have to follow the "release often" mantra; I will release it when it is ready to be released 🙂

[raven ridge]

I buy that Python is easier to parse than C++ (pretty much everything is), but that doesn't make it easier to implement

[weary garden]

when neos is released it will use CMake

[limpid forum]

well, python is backwards-compatible, isn't it? (without some low 3.x versions, I think? and of course without 2.x - that was the point of 3.x)
so they can just implement the newest one and it should parse the older ones as well

[weary garden]

@limpid forum the plan is to support the latest 3.x whatever that is when I am nearly finished with the Python schema

[grave jolt]

latest?

you mean, with pattern matching? 🙂

[weary garden]

no, I will take a snapshot of the current state of play

and update it in future in slow time

[feral cedar]

pattern matching will probably be released when you are done

[flat gazelle]

The patma isn't too insane to implement in a high level language.

[raven ridge]

It's mostly backwards compatible, but every minor version has some incompatibilities with previous minor versions.

[weary garden]

oh

[raven ridge]

lots of code broke when async became a keyword in 3.6, for instance.

[weary garden]

you mean Python pattern matching

well if that is a popular thing now I will support it

[grave jolt]

yes

Well, it's already in the language

[oblique crystal]

Popular? Would not say not yet

[raven ridge]

well, not in any released version.

it's merged to master.

[oblique crystal]

Unless you mean popular as in controversial

[flat gazelle]

What I do wonder about is how you will implement generators

[limpid forum]

hm... I thought it was 3.4+ that was correctly supported by the future versions, but I don't deal with async. so it's 3.6+ so far? or were there other things that broke?

[raven ridge]

things are deprecated or change in pretty much every minor version.

[flat gazelle]

And to a lesser extent descriptors

[grave jolt]

in an alpha version

[weary garden]

generators are not a concept specific to Python so I will create language agnostic semantic concept(s) for them that the Python schema can use

[limpid forum]

... uh, yield in C#?

[oblique crystal]

I think 3.4 introduced async and await but they weren't keyword until 3.6

[grave jolt]

generators are a concept specific to Python
nope, JS and some other languages have coroutines of some form

[limpid forum]

generators are a thing in other languages as well

[weary garden]

@grave jolt read what I wrote again - word typo.

[grave jolt]

ok

[flat gazelle]

So if you decided to support, say, prolog, you would just implement the logic engine in cpp?

[weary garden]

@flat gazelle the semantic concept library is extensible using a plugin system

[raven ridge]

https://docs.python.org/3/whatsnew/3.7.html#porting-to-python-37
https://docs.python.org/3/whatsnew/3.8.html#porting-to-python-3-8
https://docs.python.org/3/whatsnew/3.9.html#porting-to-python-3-9

[spark magnet]

i think the nuances of languages are going to make reuse of semantic concepts difficult, but we'll find out more as you implement more

[flat gazelle]

So the core idea is reuse as many already implemented language features, and just fill in whatever's missing. I do want to see how much actually ends up reusable.

[weary garden]

@spark magnet if a particular langauge poses a pathological problem than a language-specific semantic concept library plugin can be introduced.

[spark magnet]

my belief is that accurately implementing a language will always require language-specific concepts

[weary garden]

and my belief is that there is a lot of commonality between a lot of languages

[grave jolt]

well, let's wait until we get to Haskell.

[weary garden]

most differences are syntactical rather than semantical

[spark magnet]

i think you are wrong. time will tell. it's good to see you getting started on Python.

[flat gazelle]

Common lisp classes have entirely different semantics from smalltalk classes, both of which are different from python classes and java classes.

[grave jolt]

and probably C++ classes

[weary garden]

that is why classes won't be a semantic concept.

[sacred yew]

even python ints are different from java ints which are different from c ints

[grave jolt]

what even would be then...

[spark magnet]

what will they be?

[flat gazelle]

Even function calls are only somewhat portable between languages

[weary garden]

you need to appreciate that a general concept can be broken down into smaller concepts

[spark magnet]

we should stop, and let you get back to implementing. the code will explain.

[raven ridge]

java ints which are different from c ints
What's the difference there?

[flat gazelle]

Java ints define overflow, size and integer division by negative number

[raven ridge]

I didn't realize they defined signed overflow. Interesting.

[sacred yew]

c signed overflow is UB

[raven ridge]

yeah.

[grave jolt]

and Rust integers come in two flavours -- overflowable and non-overflowable

[sacred yew]

and then we have js, with doubles that coerce to ints when you use bitwises

[grave jolt]

js also has bigints nowadays...

[raven ridge]

yeah. I don't know how a language without an integer type ever got so popular.

Damn browsers...

[sacred yew]

laughs in js's many other misfeatures

php is even worse

[grave jolt]

Should I learn PHP just for the fun of it?

[sacred yew]

o_O

[feral cedar]

no

[limpid forum]

hm, for non-API stuff:
3.7 doesn't really break pre-existing code with anything else than this async/await. the other thing was erroneously accepted, so it's not really a correct syntax anyways
3.8 - I'm even surprised yield was allowed in non-leftmost for, I can't imagine how it could've been used casually. str thing shouldn't be really used for coding logic, so it doesn't affect anything that doesn't rely on it
3.9 has only changes to API
so I would say it's been pretty backwards compatible (outside the API) so far...?

[grave jolt]

Maybe I could write a compiler for a subset of Haskell in PHP.

[sacred yew]

i too enjoy suffering for no reason

[raven ridge]

It's pretty backwards compatible, but not completely.

Removed __str__ implementations from builtin types bool, int, float, complex and few classes from the standard library. They now inherit __str__() from object. As result, defining the __repr__() method in the subclass of these classes will affect their string representation.
That's a language change in 3.8 that definitely could affect the behavior of user defined classes.

[weary garden]

what I find tricky is coming up with unambiguous names for things, I want to use langauge.type.float for a arbitrary precision float but when people see float they automatically think if IEEE754 single precision.

[raven ridge]

Python's float is IEEE 754 double precision.

[weary garden]

even more confusing

although that would map to the neos concept language.type.f64

[gleaming rover]

how about language.type.fap?

unambiguously arbitrary precision 🙂

[swift imp]

lol

[sacred yew]

wait what langs have arbitrary precision floats

[gleaming rover]

well

[swift imp]

dumb ones

[gleaming rover]

I believe “arbitrary precision FLOATING point” is an oxymoron...?

[weary garden]

no, the arbitrary precision refers to the size of the mantissa

[feral cedar]

wouldn't arbitrary precision floats just fail with 0.1

[swift imp]

yeah

[swift imp]

this

[sacred yew]

excluding something like the decimal module

[feral cedar]

that's not backed by a float though right?

[sacred yew]

hmm

[raven ridge]

An upcoming change in 3.10 is

Builtin and extension functions that take integer arguments no longer accept Decimals, Fractions and other objects that can be converted to integers only with a loss (e.g. that have the __int__() method but do not have the __index__() method).
That's definitely a language change that could break user code.

[weary garden]

backed by a float?

I also have arbitrary precision rationals

[swift imp]

I've never heard of __index__

[flat gazelle]

@weary garden raku uses rat for rational types and fatrat for arbitrary precision rational

Solid names imo

[sacred yew]

fatrat?

[limpid forum]

Change behaviour but not break their logic. That's what I meant. ^^

[limpid forum]

This is interesting. It's basically saying "you shouldn't have been doing this, so we're removing this". I think someone could have more problems with older thing as it didn't error when it should. So it could break user code but it will enforce better practices (explicit is better than implicit, etc)

So it still sounds pretty backwards compatible if your logic was solid and you didn't use any tricks...?

[raven ridge]

My only point was that backwards incompatible language changes happen all the time. Only 3 years ago we had the async / await keyword mess. That's the last one I remember breaking a lot of people, but smaller things that break fewer people happen every release.

[swift imp]

is there a concept of adding custom dunders to your class?

[spark magnet]

you shouldn't invent new dunder names. Python should say what they mean.

[grave jolt]

Pylance just released a new version, and it seems like it wants to support polymorphic values now?

But, well... it doesn't work correctly

I'd be very happy if it supported this

[foggy lark]

hey

anyone here

[coral drift]

i am making test cases with the robot framework

any one experienced with robot framework

[raven ridge]

Try #unit-testing @coral drift

[coral drift]

k

where can i learn haskell

why should i learn haskell

[unkempt rock]

@coral drift Hello, this channel is for discussion about the Python language itself. You might want to ask that question on an off-topic channel.

[hearty trail]

any resources to learn python and grpc ?

[rugged lantern]

is there a standard way to let someone know what version of python your project is using?

[worldly venture]

Is this for a library or an application? If you use tools like pipenv or poetry you can do it in your Pipfile or pyproject.toml respectively, if you are using setuptools you do something like ```py
import sys
if sys.version_info < (3,5):
sys.exit('Invalid version, minimum 3.5')

in your setup.py

[worldly venture]

the gRPC docs are the best I've found https://grpc.io/docs/languages/python/basics/

[rugged lantern]

Just any project really. Cheers will look into those. And putting it in the code is a good idea too! 👍

[sacred tinsel]

you can also put it in the project's README, but using pipenv/poetry is great because it will help the developer automatically recreate the environment on their machine

if you're only using requirements.txt you can put it at the top as a comment too

if your project is on github and has a pipfile, shields.io can automatically generate a cute badge for your readme 🥰

[worldly venture]

ohhh nice

[spark magnet]

putting a version check in the code won't always work

best is to use python_requires in your setup.py or setup.cfg

[rugged lantern]

what uses the python_requires?

[spark magnet]

pip uses it

[rugged lantern]

ah cool will looking into that 🙏

[spark magnet]

the problem with a check in the code itself is that the code might not even compile to the point of being able to run the check. if you use new syntax in your file, the check will never run.

[rugged lantern]

yea makes sense. it would be better know your on the wrong version sooner too. like when pip installing 😄

[spark magnet]

One thing about the python_requires line: if someone tries to install your app on a too-old version of Python, pip might just find an older version of your code that isn't incompatible. Sometimes that's what you want, sometimes it's not.

and btw, there are tense threads about all of this in the poetry issue tracker at the moment.

[limpid forum]

You can have two underscores at the beginning to enable name mangling, but dunders as in two unders at both beginning and end are special and it's recommended you don't use them for your stuff in case Python introduces new dunders in the future. I think PEP8 mentions it in its naming guidelines

[undone hare]

Yep, the PEP8 does ask you to not do that

[knotty pond]

Hello everyone! I have made a google docs about python. It is not done, but, if you want to have a look at it, here it is. Also, feel free to make comments :)

Link: https://docs.google.com/document/d/1snPrxegKyKVxCTmAiMy1HdgYGIWfKnoojnCm5Q5DtbM/edit?usp=sharing

:D