#internals-and-peps

1 messages · Page 115 of 1

halcyon trail
#

If you're only using things like virtualenv and no docker, your environment isn't reproducible because your C based packages can be built differently, have different dependencies, they may not work without installing something using the OS package manager, etc

#

So I'd say that conda is basically a complete solution, the others (afaik), you have to have something else in the mix to make sure the native side of things is consistent and has all your dependencies

#

For just working on some personal projects on your own machine this probably doesn't really matter

paper echo
#

Conda is a bit of a weird tool, to the point where I feel like Continuum (the company behind it) doesn't understand the value of what they have created, or doesn't know how to make use of it

#

A binary package manager that can install packages in any location on the system is not a trivial accomplishment

#

And while it is tremendously useful for data science, there's no reason it shouldn't also be useful as a general dev tool

#

I have high hopes for the Mamba project

halcyon trail
#

I think they understand the value

#

That's why they released conda separately

#

We use it extremely extensively

#

IMHO if you're not going the docker route conda really seems like the way to go

#

Unless you can really guarantee full control and homogeneity over all the machines you ever run on

#

We use conda primarily as a C++ environment tool and second for python

lusty kindle
#

"here is our amazing conda, without any of the data science or bloat"

#

what more can ya ask for

distant coyote
#

hi i was wondering if anyone can help me with this code

fallen slateBOT
#

Hey @distant coyote!

Uh-oh! It looks like your message got zapped by our spam filter. We currently don't allow .txt attachments, so here are some tips to help you travel safely:

• If you attempted to send a message longer than 2000 characters, try shortening your message to fit within the character limit or use a pasting service (see below)

• If you tried to show someone your code, you can use codeblocks
(run !code-blocks in #bot-commands for more information) or use a pasting service like:

https://paste.pythondiscord.com

distant coyote
#

!code-blocks

fallen slateBOT
#

Here's how to format Python code on Discord:

```py
print('Hello world!')
```

These are backticks, not quotes. Check this out if you can't find the backtick key.

#

Hey @distant coyote!

Uh-oh! It looks like your message got zapped by our spam filter. We currently don't allow .txt attachments, so here are some tips to help you travel safely:

• If you attempted to send a message longer than 2000 characters, try shortening your message to fit within the character limit or use a pasting service (see below)

• If you tried to show someone your code, you can use codeblocks
(run !code-blocks in #bot-commands for more information) or use a pasting service like:

https://paste.pythondiscord.com

sand goblet
#

If you give a default variable a global or built-in value, is that any different that just assigning the variable in the function?py def f(local_print=print): local_print("something")
is that any different from this?py def f(): local_print = print local_print("something")
My idea of how it works is, there’s an array of local variables with the names of all the local variables, and the values which all start as null or something like that. But for default arguments, the values start as whatever object you give as the default value. Is that right, or is that not how it works?

#

Actually, could you use that even for local variables? Could you just assign all the local variables to default argument values and then the function never needs to assign any values, it just immediately has them assigned?

#

Sorry, I mistyped it earlier. Now it’s correct

peak spoke
#

They will both end up in locals, I don't think there's any difference between them beyond how the assignment is handled.

sand goblet
#

I noticed that the bytecode doesn’t have any part where it assigns the default values in the default variable functions, but I’m not sure if it still had to assign them and it just doesn’t show in the bytecode

peak spoke
#

!e It's done as a part of building the function with MAKE_FUNCTION

import dis
dis.dis("""
def f(local_print=print):
    local_print("something")
""", depth=0)
fallen slateBOT
#

@peak spoke :white_check_mark: Your eval job has completed with return code 0.

001 |   2           0 LOAD_NAME                0 (print)
002 |               2 BUILD_TUPLE              1
003 |               4 LOAD_CONST               0 (<code object f at 0x7f2aa3c60920, file "<dis>", line 2>)
004 |               6 LOAD_CONST               1 ('f')
005 |               8 MAKE_FUNCTION            1 (defaults)
006 |              10 STORE_NAME               1 (f)
007 |              12 LOAD_CONST               2 (None)
008 |              14 RETURN_VALUE
sand goblet
#

No wait

#

Instead of “LOAD_GLOBAL”, it’s “LOAD_FAST”

peak spoke
#

It will have to use load_global for the print but after that accessing local_print is the same

sand goblet
#

It’s exactly the same except for LOAD_FAST for print instead of LOAD_GLOBAL

#

!e ```py
from dis import dis

def f(x=1, y=2, z=print):
z(x + y)

def g(x, y):
print(x + y)

dis(f)
print()
dis(g)```

fallen slateBOT
#

@sand goblet :white_check_mark: Your eval job has completed with return code 0.

001 |   4           0 LOAD_FAST                2 (z)
002 |               2 LOAD_FAST                0 (x)
003 |               4 LOAD_FAST                1 (y)
004 |               6 BINARY_ADD
005 |               8 CALL_FUNCTION            1
006 |              10 POP_TOP
007 |              12 LOAD_CONST               0 (None)
008 |              14 RETURN_VALUE
009 | 
010 |   7           0 LOAD_GLOBAL              0 (print)
011 |               2 LOAD_FAST                0 (x)
... (truncated - too many lines)

Full output: https://paste.pythondiscord.com/fujovozome.txt?noredirect

prime estuary
# sand goblet I noticed that the bytecode doesn’t have any part where it assigns the default v...

The place where default values are assigned to the local variables is during the argument parsing, where it checks the tuple of default args in the function object:
https://github.com/python/cpython/blob/736ed6f7a9f465ba728198e8bca81e5fbe71bc37/Python/ceval.c#L5005
As others have described the default values are evaluated at the time the function is defined, which is the main difference between assigning it inside the function.

fallen slateBOT
#

Python/ceval.c line 5005

/* Add missing positional arguments (copy default values from defs) */```
grave jolt
#

Is there a good way to explicitly declare that a class implements a protocol? This is what I'm doing right now:

# auth.py
class OAuth(Protocol):
    async def request_link(self) -> str:
        ...

    async def submit_code(self, code: str, state: str) -> Optional[User]:
        ...

# google_auth.py
class GoogleOAuth:
    def __init__(self, config: Config):
        ...  # actual implementation

    async def request_link(self) -> str:
        ...  # actual implementation

    async def submit_code(self, code: str, state: str) -> Optional[User]:
        ...  # actual implementation

_: type[OAuth] = GoogleOAuth
true ridge
paper echo
#

At that point it's basically just a mixin or zope-style interface, no?

#

I would feel weird inheriting from it without abc semantics

#

Unless the method has a sensible default implementation I would rather have an error at class definition time than wait to have a method that inexplicably and unexpectedly returns None

#

In which case, what's the point of using the protocol at all? To maybe support misbehaving library authors who don't want to inherit from the right parent classes?

#

It would be nice if you could statically indicate that a class is expected to implement a protocol, without actually inheriting from it at runtime

true ridge
#

explicitly declaring only seems good as far as documentation goes

paper echo
#

That's kind of my point I think? The whole point of a protocol is that it's structural, not by name

#

If you are designing the interface from scratch you choose between explicitly naming the interface with an abstract base class, or just documenting the interface and using a protocol

true ridge
#

which I guess could be specified in somewhere in the docstrings, if you want to avoid inheriting (though I don't think it is likely that you'll inherit something and stuff fail on the runtime when the default method returns None, the type checker you are using would catch that misusage up).

true ridge
grave jolt
#

for performance, as the author claims

paper echo
#

I wish it was easier to extend these type checkers

#

I'm not sure if it's a fundamental issue in the way a type checker has to be designed

halcyon trail
#

That's pretty strange

#

I would expect if you inherit something explicitly and don't fully implement it then the type checker should complain

#

But re protocols it is strange to explicitly declare protocol conformance, the whole point of protocols is that you don't do that

#

ABCs are for explicit

#

And the benefits are not just documentation, there's a whole debate on whether structural polymorphism like this is actually a good idea, and in the end most languages say "no" because it's a very rare feature in statically typed languages

#

Generally interfaces aren't just about syntactic requirements, but also semantic requirements. Structural/implicit opt in is basically hoping that everything that meets syntactic requirements also meets semantic requirements

#

As opposed to being explicit which is a programmer explicitly asserting that the semantic requirements are met

paper echo
#

I do kind of like the idea of statically-checkable protocol implementation though

#

In a way it could almost obsolete ABCs

#

You don't have the MRO problems that you have with an ABC and inheritance

#

With a static checkable Protocol, the type checker would tell you whether that method is implemented or not, regardless of where in the method resolution order it is

grave jolt
#

?

fallen slateBOT
#
typing.runtime\_checkable 

@typing.runtime_checkable```
Mark a protocol class as a runtime protocol.

Such a protocol can be used with [`isinstance()`](https://docs.python.org/3/library/functions.html#isinstance "isinstance") and [`issubclass()`](https://docs.python.org/3/library/functions.html#issubclass "issubclass"). This raises [`TypeError`](https://docs.python.org/3/library/exceptions.html#TypeError "TypeError") when applied to a non-protocol class. This allows a simple-minded structural check, very similar to “one trick ponies” in [`collections.abc`](https://docs.python.org/3/library/collections.abc.html#module-collections.abc "collections.abc: Abstract base classes for containers") such as [`Iterable`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Iterable "collections.abc.Iterable"). For example:

```py
@runtime_checkable
class Closable(Protocol):
    def close(self): ...

assert isinstance(open('/some/file'), Closable)
grave jolt
#

Oh, statically checkable

#

Well, type checkers already understand protocols, it's just that Pyright doesn't do the checking if you explicitly inherit the protocol

true ridge
paper echo
#

@grave jolt yeah I would want the opposite, @typing.implements(MyProtocol)

paper echo
halcyon trail
#

It doesn't obsolete ABCs, because like I said, having to "opt in" is intentional

#

A lot of new languages of the last few years for example could have opted for structural polymorphism a la go, and have decided against it, because they don't like how it's implicit

#

I prefer it explicit as well, I think though in python given that the actual language is duck typed, there is obviously extra benefit to protocols in some cases

grave jolt
fallen slateBOT
#

Lib/urllib/parse.py lines 616 to 619

if not string:
    # Is it a string-like object?
    string.split
    return b''```
craggy night
#

Yikes @ urllib in 3.9

gleaming rover
raven ridge
#

"structural subtyping" 😄

craggy night
#

python isn't always about performance?

boreal umbra
#

it looks like that might just raise an esoteric attribute error

craggy night
#

If it's not a string, return empty bytes. I don't believe the line string.split does anything.

charred wagon
#

Well, it raises an error if the attribute doesn't exist. That's probably why that line exists

# 
>>> class C:
...     def __bool__(self):
...             return False
...
>>> c = C()
>>> from urllib.parse import unquote_to_bytes
>>> unquote_to_bytes(c)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\Mark\scoop\apps\python\current\lib\urllib\parse.py", line 616, in unquote_to_bytes
    string.split
AttributeError: 'C' object has no attribute 'split'```
craggy night
#

Ah, yea. Why not raise an error?

charred wagon
#

You mean, why not be more explicit about it? I don't know I didn't write urllib 🤷

craggy night
#

God bless

paper echo
#

Why not just catch the exception when it would naturally happen 3 lines down?

#

Or check hasattr??

#

Not like you really need to care about TOC/TOU issues (theoretically in a multithreaded environment you could assign a split attribute while this is executing.....)

paper echo
raven ridge
#

it's not wrong, it's just a strange way to write it.

#

It does just raise an AttributeError if the thing doesn't support .split - that's not completely unreasonable; that's a normal way for duck typed functions to fail if you pass the wrong type

paper echo
#

Oh i bet it's because they wanted to use the "natural" attribute error while using the fast path of returning empty bytes

raven ridge
#

yeah - if this fast path didn't exist, it would fail with the same AttributeError once it hit line 622

charred wagon
#

raising a TypeError with a proper message seems better than raising an AttributeError (maybe check if not isinstance(string, (str, bytes)): raise TypeError(...)?);

#

Response was

The AttributeError on unquote() is backward compatible with 2.6, 2.7 and 3.1. (issue 9301 is about backward compatibility)

raven ridge
#

seems pretty reasonable to me.

eager trail
#

Static methods have no access to self but otherwise have no benefit memory wise right

undone hare
#

Yep

#

Well, they have a very very minor performance benefit over normal bound functions

gleaming rover
undone hare
#

lmao

raven ridge
#

You can get the same performance benefit with regular methods by just calling type(obj).method(obj) instead of obj.method()

eager trail
#

When importing a class from another file and creating an instance of it, does it come with the imports in that file as part of the instance?

sand python
#

the entire module that the class comes from will imported, which includes all its normal import statements

magic python
#

is there a reason one wouldn't want an __init__.py in a dir which had python modules?

sand python
#

if you don't want it to be a package, for example it just has some scripts for you to use and you don't intend for the python files inside to be used elsewhere

eager trail
eager trail
#

Can't seem to phrase my Google search correctly to get an answer

sand python
#

is superclass a module here?

#

anyway, modules only get imported once, if you try to import them again you just get a reference to the one that was imported earlier

magic python
sand python
#

if you need a package you use __init__.py, if you don't need a package you don't use it

#

it has no harm in the same way giving your variables bad names has no harm, things still work but you might be making things difficult to understand

magic python
#

I'm going to assume that if that's the only example, it's fine to just use as default

sand python
#

you are turning the folder into a package

#

if you want that, do that

magic python
#

I can't see a down side from what you've said

sand python
#

the intention behind packages is to organize a bunch of modules together into one cohesive "thing"

#

and perhaps give them better names via namespace packages

magic python
#

I'm failing to see the harm with using an __init__ still 🤔

sand python
#

like, I have a scratch folder that contains like 10 different scripts that have no relation to each other

#

why would I turn that into a package? I haven't structured the files in any meaningful way

magic python
#

you're not answering why not, but i'm happy to stop this as i'm not seeing the issue

#

the variable example doesn't hold imo, or at least would need explaining how a __init__ could be as misleading as dataframe = [1,2] or something

sand python
#

not misleading

#

I was thinking more like naming everything a, b, c, etc

#

it doesn't break anything to do that

magic python
#

still doesn't hold, don't think it works as an example

sand python
#

i mean, what you're saying is "I want to make a package because I can"

#

well, you can

magic python
#

but again, if this is all there is then i'm fine with that

#

cool then 🙂

sand python
#

just as you can name all your variables a, b, c

#

if someone wishes to

magic python
#

sure, i'd have a more solid explanation of why they shouldn't in my context - as they might have in theirs for doing so

#

a,b,c might be reasonable names in some context, hence why i wouldn't say they're "wrong"

sand python
#

you know what I'm talking about though don't you

#

naming everything a letter of the alphabet

magic python
#

going to a limit case all the time doesn't make sense imo

#

or is pointless... sure - naming everything a,b,c,d... would be poor, but doesn't make sense in this case imo

visual shadow
#

Is the original discussion around "are there downsides to using empty inits?"

magic python
#

Yes - this abc thing doesn't make sense to me in that context

sand python
#

not empty inits, just inits in general

#

original question is basically "why shouldn't I just add an __init__.py to every folder with python files"

visual shadow
#

I mean, if it's a non empty init, there's no point of discussion, you need it. Otherwise it would have been empty.

#

Oh.

#

Okay, so I see two different questions.

#

Which one is the one? Question 1. Are there any issues with using empty inits in some package.
Question 2. Are there any issues with making an empty init everywhere I'm coding py files.

#

(non empty inits are, imo, by definition needed.)

#

For the former question, empty inits in a package sound perfectly fine to me. For the latter, I don't see why you'd want the hassle of making inits for throwaway scripts, it doesn't make sense.

magic python
#

Yeah - it seems fine - just wanted to check to see if there were any odd edge cases or whatever

sand python
#

anyway my point was that it signals your intention as the creator

unkempt rock
magic python
sand python
#

it might be if you're publishing things, and it might cause issues with things like setup.py package discovery

#

which looks for folders with __init__.py

magic python
#

I'm not - but if that's the case an example could be interesting, not too sure what to search there

sand python
#

they have .py files in doc and web, but no __init__.py, because those file aren't meant to be imported, they're meant to be run "standalone"

#

I don't know if adding them would actually break anything when building pandas, though

#

but for scripts that use find_packages in their setup.py it would change the result

magic python
paper echo
#

I think conceptually it's very important to distinguish between packages, namespace packages, and directories that are neither

#

doc is neither a package nor a namespace package, it's just a directory in the project tree

#

You could put __init.py__ everywhere in your filesystem if you wanted, but it'd just be confusing

boreal umbra
#

Am I right in thinking that the setup.py file is executed when you pip install something from pypi?

spark magnet
boreal umbra
#

Someone in a help session gave me the impression that something else happens and I needed a sanity check. Thanks!

#

(I host all my stuff on github because I have not yet taken the time to learn how to post on PyPI)

peak spoke
#

Does it execute when wheels are used?

spark magnet
#

it's executed on the machine that makes the wheel

raven ridge
#

assuming you're using the setuptools build backend, at least.

boreal umbra
#

I don't really understand wheels, but aeros mentioned that if I wanted to post a C module on PyPI that's available for Windows, I could build a wheel for it.

raven ridge
#

PyPI packages come in two forms: source distributions and binary distributions. A source distribution is just the source code, and when pip downloads the code it needs to build it - by executing the setup.py script, normally. A binary distribution contains an already built package - in the case of a C extension module, that means it has already been compiled.

#

and a wheel is a type of binary distribution. It basically just needs to be downloaded and unzipped, but it's already been built.

valid rose
#

poetry is awesome, it puts all dependencies in pyproject.toml, and to publish to pypi all you need is poetry build and poetry publish

terse orchid
#

How can I execute a python program with a linter that, at the end, it gives a report for type annotations like: "this variable was expected to be a float, instead got int" etc? It didn't break the program but was a different type in the runtime

eager trail
terse orchid
#

You mean manually?

eager trail
#

No, I'm guessing something about python's dynamic typing means your linter isn't evaluating correctly if you're only getting type results after the code has already run

#

I mean doing that type check dynamically at key checkpoints

#

I've not got super in depth experience with this tho, I think some of the helpers know a lot more about this

valid rose
sacred tinsel
#

I don't know whether it can give you the report you mention, but it can annotate code, so you'll be able to see from e.g. git diff what has changed

terse orchid
terse orchid
# valid rose mypy was made for that

Yes I think mypy should be able to do that but haven't figured it out. I mean running mypy main.py gives linting errors as output. What I want to do is running the program with it, and getting warnings like "variable x expected to be an int but got float" etc in the runtime with real scenarios

valid rose
#

i think you could write a decorator which checks the function annotations and emits a warning whenever the types of the arguments don't match

native flame
#

you'd also have consume any arbitrary iterator completely to check if it matches the typehint

strong imp
#

Hi! I am working on a package with some nested modules and we import some classes and functions from deep down directly into the outermost module. But in the help strings and sphinx docs they are displayed with the full module path, e.g. scipp._scipp.core.Variable. In order to abbreviate this, we can do Variable.__module__ = 'scipp'. But I am wondering if this could have bad side effects and break some code? Also does anyone know of a better solution?

prime estuary
#

There's this project, which avoids the expensiveness issue by only checking a single (pseudorandom) item each time - that way over multiple calls, it'll check the whole structure:
https://github.com/beartype/beartype/

unkempt rock
unkempt rock
#

hello guys, can anyone help me pls?

#

hello guys. i know i am the wrong place but does anyone has any hacking knowledge here..i dont want to hack anyone. i just have some important queries..

grave jolt
eager trail
#

Hey everyone, got a question regarding imports

Say I have two files, parent\__init__.py and child.py
parent\__init__.py looks like this

import some_big_library
import other_big_library

class BaseClass:
    def __init__():
        raise Exception('Implement in subclass')```

child.py needs to use BaseClass as a superclass while also keeping memory from imports as low as possible (has cost ramifications)
which of the following 3 versions of child should create the least memory requirement?
child_1.py:
```Python
import some_big_library
import other_big_library
from parent import BaseClass

class ChildClass(BaseClass):
    def __init__():
        some_big_library.stuff()
        other_big_library.stuff()```

child_2.py
```Python
from parent import BaseClass

class ChildClass(BaseClass):
    def __init__():
        BaseClass.some_big_library.stuff()
        BaseClass.other_big_library.stuff()```

child_3.py
```Python
from parent import *

class ChildClass(BaseClass):
    def __init__():
        some_big_library.stuff()
        other_big_library.stuff()```

And PS How do I test this for myself, I'm a little unclear
peak spoke
#

Modules are cached after the first import and the cost of names is negligible, just use what's the clearest and most readable

eager trail
#

I did tell him it was negligible

terse orchid
unkempt rock
#

Is there a static type checker that can already validate whether cases are exhaustive (e.g., must cover all enum values) for the new pattern matching statement in 3.10? mypy and pyright don't seem to do so yet

eager trail
peak spoke
#

I would avoid star imports as then it isn't explicit where names came from

#

And importing imported modules from a different module is definitely unusual, but if it keeps them happy

eager trail
#

True

#

I wonder if there is a pep I can point him at that demonstrates this isn't an issue

#

His background is heavily in 2.7 so I suspect this was once an issue but no longer is

sacred tinsel
#

it shouldn't be difficult to demonstrate

#

consider a.py:

import hashlib

main.py:

import hashlib
from a import hashlib as hashlib_from_a
print(f"Are the same: {hashlib is hashlib_from_a}")
eager trail
#

Brilliant, that works, ta

sacred tinsel
#

if this didn't hold, you'd be in trouble

peak spoke
#

Maybe python 1 did imports like that but even there I'd be surprised if they weren't kept somewhere after the first import

sacred tinsel
#

e.g. if you force reload a module with a type, instances of that type from before reloading will no longer pass isinstance checks against the new type

#

because the type was redefined

#

this also breaks except ErrorType checks 😔

#

for example:
a.py:

class MyError(Exception):
    ...

now in main.py:

import a
exc = a.MyError("My error instance")

try:
    raise exc
except a.MyError as caught:
    print(f"Caught: {caught}")
#

this error gets caught

# 
import a
exc = a.MyError("My error instance")

from importlib import reload; reload(a)

try:
    raise exc
except a.MyError as caught:
    print(f"Caught: {caught}")
#

this one doesn't

#

because these are no longer the same type:

unknown.png
eager trail
#

ooh

#

nice. SCIENCE

paper echo
#

is it possible to invoke importlib.reload in the main thread from a background thread?

#

use case: continuously reload a module during interactive editing

hollow hill
#

Hey guy the software inside ATM or Self-Checkout machine are about application software development or embedded system development ?

halcyon trail
#

@paper echo there used to be something that would auto reload every time it was edit, I thought

paper echo
#

ah nice

#

@verbal escarp you should see what ipython does internally ☝️

#

however ipython might be doing it single-threaded async

#

i don't think that's an option in this particular case

fallen slateBOT
#

@finite sparrow

It has arrived!

Here's your reminder: [#internals-and-peps message](/guild/267624335836053506/channel/709904092280914030/).
[Jump back to when you created the reminder](#internals-and-peps message)

finite sparrow
#

who woulda thunk

simple oracle
#

Hey guys, I wanna know if I can place custom images and text with with auto fit on a template from python

summer oak
#

Hello, I am currently looking for replacing starlette with another python api as starlette is executing one api call twice and it's fucking up with the table queries, any suggestions? How's falcon?

astral gazelle
#

sounds like a problem with your code, not with starlette

grave jolt
#

maybe the browser is making an OPTIONS request together with a POST request? Have you looked at the logs?

jaunty vessel
#

Hi All

#

anyone available who can help me with small Django question?

raven ridge
warped torrent
#

Hi, i'm having a hard problem with a program, it uses science and engineering libraries, is it the right place for that question?

paper echo
warped torrent
#

the program i'm using is openMotor (which the source is in github). I need to modify the code so it can receive a custom geometry in a special format (called spline, by the code).

warped torrent
paper echo
warped torrent
#

Hmm... Thanks 👍

summer oak
summer oak
paper echo
#

@summer oak help channels are only as useful as your question is answerable

sand goblet
#

Since you can use dict.__iter__(odict) with an OrderedDict to get a regular dict iterator from it when you just want to quickly iterate through it without caring about the order of the keys, is it right to say that the only thing you have to sacrifice with OrderedDict is some extra space? Or would it be wrong to say that?

paper echo
#

you sacrifice the {} syntax, and you express the wrong intent to your users

sand goblet
#

But I mean in the case of really wanting efficient access of the values at either end and being able to efficiently move items to either end

prime estuary
#

dict.__iter__(odict) is kinda weird, ordered dicts aren't actually the same as regular ones, they have a completely different implementation using a linked list.

sand goblet
#

Yeah

#

If you pass dict.__iter__(odict), to next then it won’t reflect if you’ve been using move_to_end because it maintains order like how you said

#

But I mean specifically in the case of that not being a problem

#

And I mean, if it was, you could always use the normal OrderedDict iterator

#

But it’s slower

#

I was thinking about what if you had a regular dictionary where you wanted to get the first or last keys, but you were frequently removing keys specifically from the beginning or end. If you do that with a regular dict then accessing from either end is O(n)

halcyon trail
#

it really depends on the implementation detail, to answer whether it's wrong to say that

#

the fact that it's the same dict iterator isn't enough information to conclude

#

if the implementation is more of a linked list approach then the data you're iterating over could be scattered in memory

#

when you construct an iterator using another iterator like that, I'm pretty sure you just "nest" the iterators, i.e. the first iterator will call next internally on the iterator you constructed it with, etc

#

that wouldn't improve perf any

#

it would make it worse

raven ridge
raven ridge
#

because OrderedDict is a subclass of dict, it's legal to use the regular dict.__iter__ for one instead of OrderedDict.__iter__.

#

!e ```py
from collections import OrderedDict
d = OrderedDict(a=1, b=2, c=3)
d.move_to_end("a")
print(list(d))
print(list(dict.iter(d)))

fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

001 | ['b', 'c', 'a']
002 | ['a', 'b', 'c']
sand goblet
#

Oh yeah. insertion and deletion would be slower since it needs to deal with connecting nodes when it does that. That makes sense

#

Also yeah it definitely uses the regular dict iter when you use dict.__iter__. You can tell because of the order. It doesn’t reflect if you’ve been using move_to_end

raven ridge
#

yep, that's what I was demoing.

sand goblet
#

Yeah since it doesn’t move the keys/values, it just reconnects nodes

raven ridge
#

I don't know what you mean by "reconnects nodes" - a dict doesn't have nodes.

sand goblet
#

The nodes in the OrderedDict

raven ridge
#

aren't used at all when you use dict.__iter__

sand goblet
#

Or is that not the right term to use there?

#

Yeah

#

Yeah it just iterates through the consecutively filled array

raven ridge
#

yes.

#

but... bypassing a class's overridden methods by explicitly calling base class methods on the derived class instance feels very icky. You're sidestepping the class's public interface. That doesn't seem wise to me.

sand goblet
#

Is there another way to do this?

raven ridge
#

This doesn't seem like a thing you should do at all.

#

The class defines how it wants to be iterated; it seems unwise to sidestep that and iterate it an entirely different way.

sand goblet
#

Okay

raven ridge
# sand goblet Okay

!e For an example of why this is a bad idea:

from collections import OrderedDict
d = OrderedDict(a=1, b=2, c=3)
dict.pop(d, "c")
print(list(d))
fallen slateBOT
#

@raven ridge :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 4, in <module>
003 | KeyError: 'c'
raven ridge
#

That's using the same trick as you're using, except instead of calling the base class __iter__ on the derived class instance, it's calling the base class pop on the derived class instance. And it leaves the derived class instance in an inconsistent state, where it thinks the key still exists, even though it doesn't.

halcyon trail
#

Even if there's no iterator nesting, it could still be an adapter internally

hearty drift
#

TIL about functools.singledispatch which mimics function overloading functionality, any thoughts about it?

halcyon trail
#

I've used it before, it's ok

hearty drift
#

Apparently only the first argument can be type hinted, so that's a bummer

halcyon trail
#

I think I ended up dropping it though, I think I needed to have more control, or slightly different behavior

#

Well, doing multi argument overloading at runtime is a bit of a bummer too :-)

hearty drift
#

mmlul fair enough

raven ridge
# halcyon trail Even if there's no iterator nesting, it could still be an adapter internally

I don't know what you mean by "it could still be an adapter" - it's just calling the superclass method. Are you suggesting that the superclass could detect when the object it's passed is an instance of a subclass, and perform subclass-specific behavior in that case? That's technically true, but superclasses aren't supposed to know about their subclasses, so that would be an absolutely insane design.

fallen slateBOT
#

Objects/dictobject.c line 3524

dictiter_new(PyDictObject *dict, PyTypeObject *itertype)```
sand goblet
raven ridge
#

yes

sand goblet
#

That makes sense

raven ridge
fallen slateBOT
#

Lib/collections/__init__.py line 120

def __delitem__(self, key, dict_delitem=dict.__delitem__):```
raven ridge
#

the OrderedDict __delitem__ calls dict.__delitem__, but also does housekeeping on the linked list.

#

if you call the underlying dict.__delitem__ directly, then the housekeeping doesn't get done, and its linked list goes out of sync with the internal dict state.

halcyon trail
#

okay, I follow you know, I didn't realize that ordereddict was subclassing dict

sand goblet
#

In CPython, don’t objects always have their memory freed immediately when their reference count is set to 0? I answered that on an exam from a couple weeks ago, and it got marked wrong. It said the right answer was “when an object’s reference count is 0, nothing references it”

raven ridge
#

iin CPython, when an object's reference count becomes 0, its finalizer runs, and then (as long as the finalizer doesn't create a new reference to it) the object is destroyed and its memory is freed, yes.

sand goblet
#

Ok, I’ll ask to be given points for that question

raven ridge
sand goblet
#

question:
What happens when an object's reference count is zero?
Options: A: The object is no longer referenced. B: The object is ready for memory allocation. C: The object is being referenced. D: The object will be deallocated from memory immediately.
It said the right answer was A.

raven ridge
#

well, depends on what you mean by "immediately"

#

immediately its finalizer is run, and as long as the finalizer doesn't create a new reference, then the object will be deallocated.

sand goblet
#

I guess immediately as in it doesn’t have to wait for some scheduled memory freeing thing to happen later

#

Like how Java does it or something like that. That’s what I was thinking of

raven ridge
#

that's true - but, because it can be resurrected by its finalizer, the deallocation isn't necessarily immediate

#

because in between the reference count dropping to 0 and the object being deallocated, the finalizer runs, and that can delay the deallocation indefinitely.

#

!e ```py
class C:
def del(self):
global b
b = self

a = C()
print(a)
del a
print(b)

fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

001 | <__main__.C object at 0x7f0454320fd0>
002 | <__main__.C object at 0x7f0454320fd0>
raven ridge
#

the reference count drops to 0 in the del a line. Once it drops to 0, immediately the finalizer (__del__) runs. The finalizer creates a new reference to the object called b, and the object is not deallocated, even though its reference count did drop to 0.

#

your teacher will probably appreciate that example, at least 😄

sand goblet
#

Alright, I see what you mean. I think I’ll accept how that question was marked now

#

Maybe if it phrased it like “if the reference count reaches 0 and stays at 0”

raven ridge
#

you can ask the teacher why the answer is what it is, and that some people on the internet told you that both A and D are correct, and see what they say.

#

Note that even if you disable the GC, the finalizer still gets called and the object still gets deallocated.

#

!e ```py
import gc
gc.disable()
class C:
def del(self):
print("destroyed")
a = C()
print("created")
print("dropping reference")
del a
print("done dropping reference")

fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

001 | created
002 | dropping reference
003 | destroyed
004 | done dropping reference
sand goblet
#

Yeah, I remember reading about that, it only uses the garbage collection cycle to deal with circular references

raven ridge
#

yep.

#

it's hard to tell if the teacher was wrong, or very pedantic. 🙂

sand goblet
#

Even if this typically isn’t something that happens, I think it still makes it so the more correct answer is A
And plus maybe saying that the class is about CPython is wrong. It just doesn’t talk about different python implementations. So maybe it’s wrong to be CPython-centric

raven ridge
#

other Python implementations don't even necessarily have reference counts.

sand goblet
#

Except idk if it’s common for memory to not typically be freed immediately in languages that use reference counting

raven ridge
#

but yes, A is definitely the most correct answer - it's basically the definition of a reference count.

sand goblet
#

And if a language is using reference counting, would it ever delay memory freeing until later after the reference count reached and stayed at 0? Or would it never make sense to do that?

raven ridge
#

it would probably mark it unused immediately upon the reference count reaching 0. Otherwise it would need to keep tracking it in some form after that, and it would prefer to forget it as soon as it can.

sand goblet
#

Alright. But still, I’m going to accept that question as it was marked now. Thanks for the help

surreal sun
#

So how do stack, pushing, and popping work in terms of parsing? when parsing through tokenized code

#

Reply to me with pings on if you know cause I gotta head off to sleep but i'm kinda curious tbh, thanks!

radiant garden
#

What's the specification for the process through which unicode identifiers are normalized, e.g. stripping diacritics and splitting some ligatures?

#

ah, NFKC

stone salmon
#

How does a function know it was called?

unborn bronze
#

Guys how can I connect a web dashboard with my bot

magic python
#

why would someone want an implicit namespace package rather than an explicit?

undone hare
magic python
stone field
#

If you mean implicit namespace packages, it allows other libraries to be imported from a namespace

prime estuary
#

Well, the main advantage is that it's then possible to have different parts of the package in different locations.

undone hare
#

Explicit namespaces usually avoid sneaky errors and shadowing

prime estuary
#

For instance maybe you have a core package, then plugins which add themselves under the namespace.

stone field
#

If I had a library called grobble I could have an implicit namespace called grobble.extra

#

and then provide my own sub-package inside it

#

but then other library authors could also use grobble.extra as the location to import their own libraries

undone hare
#

That said, I'd like to see implicit dotted imports, like Java (I think? Kotlin has that for sure). If you type os.environ.get, you don't have to import os because you start at the bottom of the import chain

#

Although I don't think it is feasible without breaking changes, due to how the import mechanism works

#

It would be a breaking change either way, actually

stone field
#

I think the flask.ext namespace is a namespace package if memory serves right

#

So other libraries than flask can use that namespace to act as the import location

#

Ah, apparently they deprecated this years ago

grave jolt
#

Is there some tutorial on how packages/namespace packages/whatever work in Python? I'm completely confused by the terminology

paper echo
grave jolt
#

is there some article that explains all of the current state of packaging?

paper echo
#

Theres another page in the python docs too

#

Python has a terminology problem: a "package" in the strict python technical sense is not what most people think of as a "package"

#

So "how to package apps/libraries for distribution" and "what is a python package" aren't the same thing

spark magnet
#

also, pypi is the "package index" which is a different meaning also

paper echo
#

The python packaging tutorial was pretty helpful iirc

grave jolt
#

god it's so confusing

#

maybe I should go with Haskell

paper echo
#

To answer the original question: a module is a namespace and it corresponds with a .py file

#

A package is a module that has a __path__ attribute and can contain other modules. It corresponds with a directory that has an __init__.py file.

#

A namespace package doesn't have the __init__.py and serves a different purpose (explained in the pep, im on mobile 😛 )

nocturne delta
#

based

shy saddle
#

please stop

nocturne delta
#

ok

lusty kindle
#

everything about python is so simple and easy other than that

paper echo
#

i don't know, imports make a lot of sense

lusty kindle
#

they do on a base level, until you get deeper. and maybe once you go even deeper they make sense again

paper echo
#

and packaging/distribution is slowly getting less fucked

#

where's the point of depth when they don't make sense?

lusty kindle
#

relative imports mostly and trying to work around your ide warnings even when the python code itself runs just fine

paper echo
#

i think if you understand the search path that kind of all goes away, no?

#

can't account for IDE warnings though

lusty kindle
#

the warnings are half of what makes it so hard

#

and sure, thats not python itself but if its vscode or pycharm or another major program we all use, lol

#

'clearly this import wont work it is warned, lets try other stuff'

#

5 hours later, ah that warned import was it after all

#

😛

paper echo
#

lol i can't account for vs code being wack

#

pycharm mostly does the right thing although in general it's a very powerful tool that is not really beginner-friendly

#

i've definitely noticed a trend of people asking about IDE warnings without ever trying to run the code

lusty kindle
#

all i know is personally, ive seen a lot of very strong programmers get held up and annoyed trying to import at times. for something as simple as an import its funny

paper echo
#

it's almost like dynamically-typed languages aren't beginner friendly

lusty kindle
paper echo
#

strong programmers coming from other languages, i could see struggling. most languages i think lack a clear, "outsider-readable" explanation of "how all this stuff fits together"

#

if anything, python has unusually good docs in this regard

#

how does ruby search for gems? how does maven work? good luck finding that info

peak spoke
#

Can't say I ever had problems with the ide not understanding relative imports and importing from packages, usually it's just a misconfigured project root

paper echo
#

python is comparatively simple, i think:

  • python itself has a default search path including various site-packages directories
  • setuptools/flit/poetry extracts contents from a package archive (tarball, wheel, whatever) and dumps them in site-packages
  • pip fetches the archive from pypi and installs dependencies
  • venv creates a python executable that has all the right search paths set up internally for self-contained use
sacred tinsel
#

I've had a problem with PyCharm recently not getting it completely right with imports but it was a more complicated (nested) project structure

paper echo
#

most programming languages don't even have a venv equivalent...

sacred tinsel
#

i.e. monorepo basically with 2 separate apps

#

but it didn't fail catastrophically

paper echo
#

i have that situation at work (except its like 10+ apps and not just 2) @sacred tinsel , i have a separate pycharm project for each

#

it's not ideal

#

i'd really like to be able to configure different interpreters for different content roots, since each app has its own venv

#

not sure if that's possible in pycharm

sacred tinsel
#

omg you can!

#

and it works really well

paper echo
#

do tell

sacred tinsel
paper echo
#

👍

twin sky
#

Is the logic true?

unknown.png
magic python
#

I'm passing around a module, that has a class instantiated within it, and then accessing that class. I'm wondering if this is bad practice at all? The reason i'm wondering is that mypy gives error: Module has no attribute when it's doing mod.class_instantiated_within_module.value, not sure what to search for though

halcyon trail
#

I'm just curious, why are you passing around a module?

magic python
#

module has metadata within it

#

might not be the best solution, it's the thing that's happening atm though 😅

halcyon trail
#

Okay, but metadata is data ultimately, right, so to pass around data you'd typically use an instance of a class

magic python
#

Yes - i'm not sure what you mean here though - there's an instance of the class created within the module, which is then imported

halcyon trail
#

and when you say "class instanted within it", you are basically saying that the module creates a global of some class?

magic python
#

I have something like:

# metadata.py
x = 1
y = 2

other = Other(x = x, y = y, ...)

along these lines, and then elsewhere I have

from ...metadata.data_id import metadata

this sort of pattern

halcyon trail
#

I'm saying that usually (there are rare exceptions I'm sure), you don't "pass around" modules themselves, e.g. as arguments to functions. Typically you'd just pass around instances of objects, occasionally you'd pass around types.

#

Okay, sure, so you have a global

magic python
#

@halcyon trail makes sense, i guess i'll ignore all errors here then

halcyon trail
#

Well, that wasn't what I was saying 🙂

#

Probably you should just be passing around the other instance itself

magic python
#

that's my solution 😎

#

for this it's probably not worth fixing - but instead can think of better approach in future if this is nonsense

#

but idk how you'd usually do it - if you have module level vars, and classes, then passing around a module kinda makes sense

halcyon trail
#

Need a bit of a broader context to answer that question. Using globals isn't usually the best but if it's some kind of fixed, unchanging data then it's often fine

magic python
#

maybe it should be a class instead of a module, seems kinda semantics tho

halcyon trail
#

why does it make sense to pass around the entire module

magic python
#

why doesn't it? It's a module that has metadata info, i don't understand why it wouldn't make sense

halcyon trail
#

when you say "pass around", are you saying you have a function def foo(x): print(x.other) and you call it foo(metadata) ?

magic python
#

not saying my lack of understanding carries weight here - just saying that I don't really see why it would be an issue 🤔

magic python
halcyon trail
#

what does "passes it to another" mean

#

passes it to a function?

#

because "another" there can only refer to "another module", and you don't "pass" things to modules

magic python
#

passes to a function , yes

# metadata/data_1/metadata.py
x = 1
other = Other( ... )

# mod1.py
from metadata.data_1 import metadata
from . import mod2

mod2.something(metadata = metadata)

something along these lines

#

if that's not clear it's probably not worth making clear tbh tho

halcyon trail
#

it could be fine in rare cases. if you pass a module as an argument to a function, then the assumption is that you have other modules that have exactly the same interface, otherwise obviously there's no point making this an argument ot a function

#

But, the fact that these modules have the same interface isn't expressed

#

That's the point of classes and instances, when you write a class it expresses a certain interface, and now you know that every instance of the class has the same interface

#

In this case, it seems like you're just passing data around, the way you'd usually do that is to have a class that holds the data you want, and then passes instances of that class to functions

magic python
#

i just don't get the difference here i guess

#

instead of doing that I'd wrap everything in the module in a class?

halcyon trail
#

So Other is a class

#

You know that all instances of Other will have the same attributes

magic python
#

I know what a class is - i don't understand why there's a meaningful difference in use between things here, the metadata is metadata for a particular data set

halcyon trail
#

if mod2.something takes an instance of Other, then it's clear what interface you're expecting

magic python
#

basically the suggestion seems to be that I would instead make a MetaData( ) class, and create an instance of that which has Other as a subclass or something?

#

iiuc

halcyon trail
#

If your module has more data than Other, then yes

#

not a subclass

#

a member

# 
@dataclass
class MetaData:
    x: int
    other: Other

or something like that

magic python
#

so I'd have MetaData with class vars which have the values which are currently globals within the metadata.py module 🤔

halcyon trail
#

not class vars

#

MetaData is a class

#

you have instances of MetaData

magic python
#

right fair, and if i have different vars for different data I'd have to add to this class it seems

#

which is fine i guess,

#

sorry @halcyon trail , i am wondering, given :

#metadata.py (old)
from . import project
x = 1
other = project.Other()

i can convert this to a class with:

# metadata.py (new)
class MetaData:
    x : int
    other = project.Other

metadata = MetaData(x = 1, other = project.Other() )

then, before I've been doing:

from .metadata.data_id import metadata
metadata.x
metadata.other.attribute

the usage now would be what 🤔

halcyon trail
#

I'm not quite sure about the nesting levels here since you named both your file and your variable metadata

magic python
#

ah ok - can change them to make this clearer that's fine, sorry

halcyon trail
#

that said, you wouldn't define MetaData in the same place where your global is defined

#

part of the whole point here is that you could define MetaData at a higher level, and then reuse MetaData class to define metadata instances in multiple files

#

defining that class ensures that you are defining all the metadata attributes you need

magic python
halcyon trail
#

np

grave jolt
#

!rule 6 @fathom snow, we don't allow any advertisements here.

fallen slateBOT
#
Rules

6. Do not post unapproved advertising.

ember lava
#

has anyone used the real python membership? is it worth it?

static bluff
#

O.o

#

I feel so directionless

#

Which is weird. I almost always have a sort of 'pull' that always keeps me moving forward

paper echo
#

does anyone know why isinstance(np.array([1,2,3]), Sequence) is False?

flat gazelle
#

it doesn't have .count, .index etc

paper echo
#

i wasn't sure how that worked - is the presence of those "mixin" methods required?

fallen slateBOT
#

Lib/_collections_abc.py line 1015

class Sequence(Reversible, Collection):```
flat gazelle
#

yes, it is implied that Sequence also has count and index. Typecheckers rely on this for example

paper echo
#

i see, interesting

#

that seems backwards to me

flat gazelle
paper echo
#

inheriting from Sequence should grant those methods perhaps, but they seem like odd requirements for the protocol

#

i guess it's just not a good fit for the type hierarchy envisioned in collections

flat gazelle
#

pretty much

paper echo
#

fortunately numpy as np.typing.ArrayLike now

flat gazelle
#

a good point raised was also 0d arrays

paper echo
#

0d arrays are cursed though

#

hyperopt uses them internally iirc

#

unfortunately ArrayLike isn't generic 😦

#

what's the right way to handle this

def get_label_mapping(data_labels: ArrayLike, given_labels: ArrayLike) -> Mapping[_S, _T]:
    data_labels = np.asarray(data_labels)
    given_labels = np.asarray(given_labels)

mypy correctly knows to specialize the type of data_labels, but it's still just np.ndarray, not np.ndarray[_S, DTypeOf_SWhateverThisIs]

#

maybe this is running into limits of what you can infer statically

#

huh, it looks like numpy.typing.NDArray doesn't actually exist

#

weird...

#

!e ```python
import numpy as np
print(np.array(10))

fallen slateBOT
#

@paper echo :white_check_mark: Your eval job has completed with return code 0.

10
paper echo
#

they exist. why? who knows

#

but i know at least 1 library that relies on them

grave jolt
#

hmmmmmmmmmmmm

# 
-    return product(*map(enumerate, iterables))
+    return product(*map(enumerate[_T], iterables))
``` this fixed it
swift imp
#

Isn't numpy working on getting type hints for size, shape and type for arrays ?

#

Or am I misremembering

gleaming rover
#

but size/shape would mean dependent typing, which Python doesn't (really) support

grave jolt
#

just marking the size isn't really dependent typing

#

you can do it to some extend in normal Python, with Peano integers, and you can even to things like +1 or -1

paper echo
#

I would settle for having to manually cast when changing sizes

#

Also I think annotating number of dimensions is more generally useful than annotating the specific shape

#

Again, I don't mind if I have to cast manually after adding a dimension

#

That said, i have messed with writing an NDArray interface in Idris

#

The ergonomics for the end-user are there, but there is a lot of burden on the library author to actually do all the dependent types correctly

halcyon trail
#

what do you mean by marking the size?

#

and I tend to agree, that tracking rank is generally going to be more useful, and more doable, than tracking specific dimensions

#

although, broadcasting tends to be fairly magical, would be interesting to see whether and how that could be made to work

gleaming rover
grave jolt
#

you can't even do addition, really

halcyon trail
#

you can do it I guess if you define all the types by hand

lilac yew
#

I dont know whether it has been discussed or not, or even it is relevant to this channel. But wanted to know your opinion about this repo

lament sinew
lilac yew
lament sinew
#

no, just talking from expperience

#

these lists are opinionated, so there'll always be disagreement

gleaming rover
lament sinew
#

anyone who thinks an exception is more readable than an if is completely out of their mind

lilac yew
brave badger
#

The example that they had provided in https://docs.quantifiedcode.com/python-anti-patterns/maintainability/returning_more_than_one_variable_type_from_function_call.html isn't exactly compelling. There's plenty of other cases where it makes more sense to return a None type than raise an exception.

If user input influences the result of the function whatsoever then it's only proper to raise a specialized exception, but if it's some get_cached_credentials method on some cache handler service object in the guts of your code, None makes sense.

Even then, in typed Python, one could argue that a Union of types is a distinct type itself, and that isinstance checks or (future) type guards are a way to refine that type into something specific.

valid rose
#

this might sound dumb, but is there a way such that when you import a specific module, that modules modifies the current module that its being imported in first with the interpreter executing the code

#

an example is ```py
import preprocessor
#define PI 3.141
print(PI)

#

this mght be stupid

lilac yew
modern night
#

you can then use inspect to read the source and do what you propose but idk about how you would do this for files that aren't __main__

valid rose
#

but this would lead to issues if the file imported is itself being imported by another file that is being imported by yet another file

#

if that makes sense 😅

modern night
#

I don't think this is a good idea, I was just saying how to do it 😅

valid rose
#

anyway, the __main__ idea is nice

#

ill try to implement it

modern night
#

whats wrong with pi = 3.141?

valid rose
flat gazelle
valid rose
#

im not going to use this in actual projects

modern night
#

cool idea tbh though, would be fun to mess around with

flat gazelle
#

return value errors vs exceptions are a bit more debatable, but IO must always be try and see if it fails

valid rose
#

macros bad is engraved into my mind

modern night
#

you could look through __main__ for all imports, then recursivly walk through each file that those files import until you hit a file with no imports, but thats just an idea, not sure how well that would actually work

flat gazelle
azure fiber
#

Hi, Why I am sometimes getting error with this
from .schemas import schema
and sometime with this

from schemas import schema```
flat gazelle
#

you are running your program in 2 different ways

azure fiber
#

No,

#

Same venv I am using

flat gazelle
#

what command do you run to start the script?

azure fiber
#

uvicorn main:app --reload

#

I am running fastapi

#

I have developed so many project in django, but didn't receive any error like this , I run few project in fastapi too. but doesn't recieved this type of error

#

I am receiving it first time

flat gazelle
#

essentially, the difference is that . imports require a parent package, so if you have a tree like

project root
\-main.py
\-package
  \-a.py
  \-b.py
```then in a.py, you can do `from . import b`, but you can't do `from .package import a` in main.py, you would have to do `from package import a`
azure fiber
#

My structure is as follows \-main.py \-models.py \-schemas.py \-settings.py

#

But still what is working currently is import settings in my models.py

#

from . import settings is giving an error

#

Yesterday, from . import settings was working

#

But , import settings was giving error

flat gazelle
#

you probably changed how you are running your project, if you do uvicorn project_root.main:app, main now has a parent package, so . imports will work, and the path is one level up, so absolute imports will not

magic python
#

I've heard using Any is lazy, in type hinting, but I'm not sure how one would type hint this:

def f(
    fn: Callable,
    kwargs: dict[str, Any],
) -> Any:
    return fn(**kwargs)

without using Any

flat gazelle
#

you can use a typevar for the return value, but yeah, sometimes the type system just can't express it

magic python
flat gazelle
#

Any is just a type

magic python
#

ok hm, i'm not sure what this would look like with a typevar instead of a type then i guess

flat gazelle
#

a TypeVar would look like

T = TypeVar('T')
def f(fn: Callable[..., T], kwargs: dict[str, Any]) -> T
magic python
#

oh ok, where do i find doc on this, and what's the advantage here?

flat gazelle
magic python
#

not sure what T is, but we're saying - we're passing some callable with variable amounts of T, and then we're returning T

flat gazelle
#

it means, given a Callable return any specific type T, we return that type T

magic python
#

oh, Callable[ ..., T] means - variable amount of args, and return type T, ok cool

tidal cave
#

people said that the best way to learn code is doing some projects

#

but if i dont know how to code the projects is it okay to look at someone code tutorials (like how to make hangman game in python)

paper echo
#

Important for checking file existence in particular

#

Not important in all or even most applications, but I believe that is why it's encouraged

#

Oh lakmatiol already said it 😛

lament sinew
#

it's uncharitable of you to think that i meant catching exceptions is always bad, independent of context.
i disagree with with eafp as a philosophy

#

being trigger happy with try/excepts will bite a programmer in the ass a milliion times before they ever find thenselves with a race condition like this

grave jolt
#

I think EAFP doesn't mean 'exceptions'. For example, if open returned either a file handle or an error value, it would still be thread-safe.

boreal umbra
#

Had the keyed indexing pep been approved, something I wanted to make was a type annotation system for pandas, but the more I learn about the library, the more futile such a thing sounds.

#

I understand numpy is doing something similar, but I don't think there's much you'd need to represent other than the data type of a given array.

halcyon trail
#

I think it's pretty futile because most schemas in pandas only become known at runtime

#

and the point of type annotations is mostly to do checking statically

boreal umbra
#

Perhaps, though, Python's typing system is what makes something as flexible as pandas a possibility. Is R typed?

halcyon trail
#

R is dynamically typed

#

but it doesn't really matter, you can have dynamically typed things within a statically typed language

#

At the end of the day pandas' dtypes and schemas are practically a mini type system unto themselves

#

Where I work, for example, we have concepts that parallel pandas dataframes in C++ (with far less functionality of course), and it all works just fine despite C++ being statically typed

swift imp
#

I think being able to annotate what columns, dtypes, and index levels you expect would be pretty useful

#

I guess me saying shape was poor choice

#

But if you're specially slicing out a column called x being able to tell if that column exists would be incredibly useful info to get from static analysis

spiral dagger
#

What is "foo" in Python?

rich cradle
#

foo and bar are just placeholders used in various programming examples. They're not keywords or anything like that.

paper echo
#

e.g. "i want to catch TypeError but only when some_package.foo raises it"

#

also for what it's worth python nowadays typically doesn't require much EAFP, you can ask getattr/.get/next to return a default value instead of raising an exception

# 
class NullContext:
    def __enter__(self):
        return None
    def __exit__(self, *args):
        return


def open_safe(*args, **kwargs, on_failure=None):
    try:
        return open(*args, **kwargs)
    except FileNotFoundError as exc:
        if on_failure is not None:
            on_failure(args, kwargs, exc)
        return NullContext()


logger = logging.getLogger(__name__)
with open_safe('something.txt', on_failure=lambda _,_,exc: logger.exception(exc)) as fp:
    if fp is None:
        print('Oops, file not found!')
halcyon trail
#

I don't think it was ever about requiring EAFP, just that it's the mentality of the language

paper echo
#

i think what i'm saying is that most python devs don't really "live by" EAFP except where necessary for toc/tou issues

#

it's ugly and awkward and you mostly don't need to do it

halcyon trail
#

idk a quick google search shows dozens of style guides, articles, etc recommending it

#

This is a language that uses exceptions to stop for loops, after all

paper echo
#

i agree that it's silly to catch IndexError instead of just checking the list length

#

basically an optimization

halcyon trail
#

maybe that blog post says that's what EAFP is specifically for but if you read around broadly you'll see tons of places recommending EAFP more broadly than that

paper echo
#

i believe you

safe hedge
#

I think use of .get/getattr etc is really dependent on what you intend to happen if something isn't found

halcyon trail
#

the thing is that python doesn't have any error handling, or mechanisms to make transformations of data easier, outside of exceptions

#

so often the alternative to catching is worse

#

Here's a good example from a quick google

# 
if text.isdigit():
    num= int(text)
else:
    num= None
#

IMHO, this is bad code

safe hedge
#

It's also flawed I was thinking of is_numeric

halcyon trail
#

you're using two different functions here, and assuming that isdigit() true is good enough to guarantee that int doesn't throw

#

it's even possible that assumption is true, but I still think this is bad

#

the try-except version is better

peak spoke
halcyon trail
#

another example is trying to set something that is nested a few layers deep into a dict, if all the keys are present, else do nothing. It's a lot nicer to do this with exceptions rather than to keep nesting if ... in ... checks

#

Ah good, TIL 🙂

#

Personally I would prefer a result type, you could just write num = test.toInt().getOrDefault(0) or something like that, would be much nicer, but it's not the python way

#

or sorry, I guess getOrNone()

paper echo
#

!e ```python
def with_handlers(*exc_handlers):
exc_types = tuple(t for t, _ in exc_handlers)
def do(func, *args, **kwargs):
try:
return func(*args, **kwargs)
except exc_types as exc:
for t, h in exc_handlers:
if isinstance(exc, t):
return h(func, args, kwargs, exc)
return do

def constant(val):
def func(*args, **kwargs):
return val
return func

text = 'hello'
print(with_handlers((ValueError, constant(None)))(int, text))

fallen slateBOT
#

@paper echo :white_check_mark: Your eval job has completed with return code 0.

None
paper echo
#

"we have composability at home"

halcyon trail
#

so ugly

#

I'm glad that so many languages are now big on allowing left-right transformations of data, really makes a huge improvement to readability

paper echo
#

it's only ugly if you use python and not hy 😛

#

what's a left-right transformation?

#

!e ```python
def with_handlers(*exc_handlers):
exc_types = tuple(t for t, _ in exc_handlers)
def do(func, *args, **kwargs):
try:
return func(*args, **kwargs)
except exc_types as exc:
for t, h in exc_handlers:
if isinstance(exc, t):
return do(h, func, args, kwargs, exc)
return do

def do_stuff():
yield 1
yield 2
return 'hello'

gen = do_stuff()
gen.send(None)
gen.send(None)

print(with_handlers(
(StopIteration, lambda f,a,k,e: int(e.value)),
(ValueError, lambda *args: print('that is not an integer')),
)(next, gen))

fallen slateBOT
#

@paper echo :white_check_mark: Your eval job has completed with return code 0.

001 | that is not an integer
002 | None
paper echo
#

is there a straightforward way to get the function that triggered an exception from its traceback?

halcyon trail
#

I just mean that the data goes from left to right. Things are executed in the order you read them.

#

like what I wrote above

#

num = test.toInt().getOrNone()

paper echo
#

ah, yeah

#

also elixir/f# pipes and clojure threading macro

#

UFCS is a thing too

halcyon trail
#

I think that's the first reason why your version is hard to read. The second reason is because once exceptions are involved, you don't just get a value that you can act on, you need to pass lambdas in and catch stuff, etc

#

I'd rather start with a value and then call getOrThrow() if I need to

#

I feel like you (or somebody else?) has mentioned UFCS as well but it's a very rare feature

#

but yeah, pipe operator and extensions are probably the two most common ways to do it. And in lisps of course it's trivial to do with a macro.

#

But anyway, but given how python doesn't have any left-to-right support, it doesn't have any support for a Result type, it doesn't have null aware operators, etc... the alck of all these features ends up making exceptions a lot more appealing than they might be otherwise, which is what leads to EAFP, broadly

#

(I think)

paper echo
#

i agree that the main limitation is syntax, not necessarily language semantics

# 
(print (with-handlers ((StopIteration (fn [f a k e] (int (. e value))))
                       (ValueError (fn [f a k e]
                                     (print "that is not an integer")
                                     None)))
  (next gen)))

a hypothetical hy interface to the same functionality might look like this, which i think is a lot more readable

#

with and try fundamentally are not composable, and lambda is very restrictive

lament sinew
paper echo
# 
handlers = [
    (StopIteration, handle_stopiteration),
    (ValueError, handle_bad_int),
]
with handling(*handlers):
    next(gen)

@halcyon trail meanwhile this does not look bad to me

paper echo
#

once in a while we get help questions like this... "why doesn't my db insert work", "why doesn't my discord bot respond to messages"

visual shadow
#

If you use a bare except and return None, how will you ask forgiveness for what you've just done? 😛

halcyon trail
#

@paper echo i don't even know what that code does

#

to me that looks pretty bad

paper echo
#

which, the hy version?

#

or the handlers = version?

halcyon trail
#

handlers =

paper echo
#

it's hypothetical, but could be easily adapted from the with_handlers that i wrote above

halcyon trail
#

I'm also not really sure what the purpose is here, how we ended up with stop iteration

paper echo
#

just demonstrating how you could theoretically make exception handling and EAFP style more composable and functional(-looking) without losing the good aspects of it

#

consider writing a function just to wrap something in try/except and return a "result" type. this is a generalization thereof

halcyon trail
#

the question is what have you actually gained over try/except. I guess the overall thing is an expression, that's something

#

but when its' so ugly you have to factor out separate "handlers" variables, I don't think it's a win in any other way

#

but it's kind of a strange example because there isn't any compsition that I can see, I just see two separate handlers for the same input stream

paper echo
#

being an expression means it's fundamentally composable, and now this chunk of handlers is something you can re-use

#

well, more composable than try/except

#

you could use it as a decorator too

#

i'm not arguing that you should all go start using this

halcyon trail
#

what are the "good aspects" of exception handling/EAFP style?

paper echo
#

avoiding the toc/tou problem

#

and generally interacting with external resources, where you don't have a cheap way to check if the resource is available until you try to use it

halcyon trail
#

you don't need exceptions for that either though

paper echo
#

right, you can return some kind of result type

halcyon trail
#

and toc/tou is only relevant in rare cases

paper echo
#

and you could use this with-handler pattern to wrap stuff that raises exceptions by returning a result type instead

halcyon trail
#

your example of handlers above would be handled more nicely by sum types + pattern matching

#

Yeah, you could

#

but it's all kind of a workaround, the obvious thing is just to start with the result/sum types 🙂

paper echo
#

absolutely plus1 on sum types + pattern matching

halcyon trail
#

It's much easier to start with a result type, and if you don't plan to deal with errors locally, .getOrThrow(), than to wrap exceptions back into results/values

paper echo
#

good point, but it is interesting that python gives you just enough tools to bolt on that kind of functionality

#

hell, you could literally just return ValueError if you wanted

#

get_or_throw is then almost trivial, and 3.10 lets you pattern match instead of using if isinstance

halcyon trail
#

yeah, will be interesting to see if/how pattern matching shakes things up, will there finally be a really widespread Result library in use, etc

#

I haven't had much time to think about it

paper echo
#

wouldn't a "result" type basically be

Result = Exception | Any
#

well, maybe Result[T] = Exception | T. not sure how to write that properly in python

halcyon trail
#

something like that, yeah, but realistically you'd probably want Result to be a proper class

#

so you can give it member functions

lilac yew
halcyon trail
#

although, I don't know if you can then match directly on a result, is there any kind of "operator overloading" for pattern matching? Probably not. So you'd have some kind of public attribute that you match on

# 
class Result(Generic[T]):
    data: T | Exception
#

etc

#

Also you probably also want at least the option of a Result type that is also typed on the kind of error

#

not to always erase it

paper echo
#

i was about to say, you can get checked exceptions with this

#

@lilac yew nuitka has been around for a while, never used it though

#

some people on hacker news said that it was modestly faster than cpython and worked pretty well for distributing standalone binaries

lilac yew
#

Sometimes i think , isnt it possible to rewrite the python lang to run to complie in machine code. do you think the creator(Guido van Rossum) who said he gonna make it faster, is actually trying to convert it into machine code excutable?

#

If it is faster then cython then why it is not so popular? strange..

paper echo
# 
# Ad-hoc sentinel
class Nothing:
    pass
nothing = Nothing()

E = TypeVar('E', bound=Exception)
T = TypeVar('T')
class Result(Generic[E, T]):
    def __init__(self, value=nothing, exception=nothing):
        self.value = value
        self.exception = exception

unsure if there's a way to allow E to be a Union[Exception1, Exception2] , or if that "just works"

radiant fulcrum
paper echo
# lilac yew Sometimes i think , isnt it possible to rewrite the python lang to run to compli...

isnt it possible to rewrite the python lang to run to complie in machine code
pypy and graalpython both do this, i believe. but with just-in-time compilation, not ahead-of-time

is actually trying to convert it into machine code excutable
unlikely, that'd require a bottom-up rewrite of cpython

cython
cython is not cpython. also library compatibility i think has been questionable in the past; not always clear what will and won't break with something like nuitka, even with the best intentions of the developer. and i think relatively few devs care about distributing self-contained python binaries, compared to people writing web server backends, command line tools, machine learning libraries, etc.

#

cython is actually a great example of what cf8 said: i usually see a 2x or more speed up by literally just copying and pasting my pure python code into cython. but do you want to do that all the time? definitely no.

radiant fulcrum
#

can you do that all the time? no

#

Especially in the world of IO

paper echo
#

it would however be really really cool to have multiple up-to-date well-maintained python implementations, not unlike common lisp. use cpython for dev and interactive stuff, compile with nuitka to ship to the end-user, or run on the server with pypy or graalpython.

radiant fulcrum
#

but equally in the world of data science you cant really get much faster than what python already has in it's ecosystem

#

considering it's mostly written in highly optimised compiled languages

halcyon trail
#

I mean, depends what you're talking about, but that's not exactly true

#

a + b + c in python, even if a, b, c are numpy objects and + goes straight to machine code

paper echo
#

i agree it's not necessarily true. but more importantly data science work is heavily interactive. the ml/data engineers however could definitely benefit from having a faster python runtime when trying to make things go fast.

halcyon trail
#

is still far less efficient than a + b + c in a compiled language, or probably even in julia

radiant fulcrum
paper echo
#

even pypy has substantial function call overhead and doesn't appear to be that aggressive about inlining (in my limited testing)

paper echo
true ridge
#

it would however be really really cool to have multiple up-to-date well-maintained python implementations, not unlike common lisp. use cpython for dev and interactive stuff, compile with nuitka to ship to the end-user, or run on the server with pypy or graalpython.(edited)
Indeed, it would be though imagine the complexity behind it. Even though they would share some common aspects (e.g the standard library, or the basic reference algorithms for features) it is a really long/hard process to catch up with CPython. Also you have account for that, they will always have to follow one release back since even in betas a lot of internals might change.

radiant fulcrum
#

it's a JIT compiler that links data science libraries to avoid callbacks to Python

halcyon trail
#

I think in terms of the basic design of the language, if you were designing a data science language from scratch, doing what Julia did is simply the obvious thing to do

paper echo
halcyon trail
#

I'm not sure where Julia is in terms of ecosystem though, other language decisions, etc

paper echo
radiant fulcrum
paper echo
#

julia's big problems right now: sloooow jit warmup and a bad case of NIH syndrome when it comes to tooling

#

the former is actively WIP and has improved a lot

#

the latter, idk

halcyon trail
#

Yeah, I've read a bit about problems in Julia

#

the basic premise though of using LLVM as a backend is somewhat hard to argue with though

#

at least, assuming the warmup issues can be overcome

paper echo
halcyon trail
#

what's kind of interesting is whether you can build a nice language for data science that's statically typed or not, I'm not really sure, all the repl's for static languages I've used had some limitations, but maybe they can be overcome, I'm not sure

paper echo
#

i think julia-the-language is wonderful, and the library ecosystem is growing very fast and iterating towards what i think is a viable python replacement for researchers. but there is no scikit-learn equivalent yet.

#

julia could be a good "general purpose" language too, at least from the perspective of its language design. not sure how the runtime does with high concurrency stuff like a webserver.

lilac yew
true ridge
#

One thing to just keep in mind is that, let alone the advanced ones you can't even do any of the fundamental compiler optimizations on the bytecode compilation step. The most you can do is just a bit of constant folding. For doing advanced stuff (e.g common subexpression elimination, loop invariants etc) you need to record traces in the runtime, with known types and codepaths and then you can do the optimizations.

halcyon trail
#

Is it clear to anyone why this script hangs:

import subprocess


def main():
    p = subprocess.Popen('/bin/bash', stdin=subprocess.PIPE, stdout=subprocess.PIPE)
    p.stdin.write(b"echo cmd\n")

    if p.stdout:
        print(p.stdout.read())

    print("done")


if __name__ == '__main__':
    main()

I thought maybe because of buffering, but pasing bufsize to Popen doesn't help

#

If I eliminate the stdout=subprocess.PIPE argument, it runs as expected (and of course it will skip the loop since p.stdout will be None)

grave jolt
#

@halcyon trail Doesn't /bin/bash open a REPL? i.e. it asks for input lines repeatedly.

undone hare
#

Yes, that's the issue

halcyon trail
#

Not sure I follow, why does it hang on the stdout call as a result?

grave jolt
#

🤔

#

If you do p.stdin.close() after p.stdin.write(), it suddenly works

halcyon trail
#

that makes sense, actually

#

that explains why it works when you stop capturing stdout

#

when you hit the end of the parent process, stdin closes

#

and then it probably flushes stdout at process end

#

oh, maybe it's stdin that needs to be flushed?

grave jolt
#

p.stdin.flush() doesn't work

halcyon trail
#

nope

#

very strange

#

there should be some way, to keep a process alive, and send it stdin, and receive stdout

#

if you close stdin that way I don't think you can reopen it

undone hare
#

Closing stdin will cause bash to exit

grave jolt
#

I'm confused as to why flush doesn't flush the stdin

undone hare
#

It does, bash just doesn't exit

# 
akarys@mojito ~> bash
[akarys@mojito ~]$ # I'll press CTRL+D to close stdin
[akarys@mojito ~]$ 
exit
akarys@mojito ~> ```
halcyon trail
#

bash doesn't have to exit

#

I wouldn't care if the script still hung

#

but why does it hang before printing out cmd

#

that's what's confusing

#

well, I guess there's nowhere else for it to really hang 🙂

#

After you do p.stdin.write and p.stdin.flush, you would expect that there would be something available to read on stdout

#

okay, version that works

# 
import subprocess


def main():
    p = subprocess.Popen('/bin/bash', stdin=subprocess.PIPE, stdout=subprocess.PIPE, bufsize=0)
    p.stdin.write(b"echo cmd\n")
    p.stdin.write(b"echo cmd2\n")
    p.stdin.flush()

    if p.stdout:
        for line in iter(p.stdout.readline, b''):
            if line.startswith(b'cmd2'):
                break
            else:
                print(line)

    print("done")


if __name__ == '__main__':
    main()
#

for the curious

peak spoke
halcyon trail
#

right

#

I eventually realized that

paper echo
#

@halcyon trail there are a bunch of warnings in the docs about deadlocking due to reading and writing to a subprocess, i think they recommend using .communicate

grave jolt
#

but communicate waits until the process ends.

paper echo
#

if you need to interleave reading and writing, you basically need to read and write in 2 different threads

paper echo
#

i think i've encountered that before

halcyon trail
#

you only have to use threads if you want to be writing at the same time as you're also reading. If you're fine with writing, then reading what's available up until either some kind of sentinel , or until a timeout

#

you don't need threads

#

but that is a way to do it, for sure

paper echo
#

what i'm wondering is whether it's possible for the subprocess (in this case bash) to cause things to deadlock. is it possible that p.stdin.write could get stuck blocking forever for any reason?

halcyon trail
#

my guess would be yess

#

if you keep writing in, but don't empty stdout

#

maybe the bash subprocess will stop, waiting for stdout to be empty'ed

#

but I'm not really sure

paper echo
#

that was what i was wondering, i don't know either

halcyon trail
#

I really need to read my copy of Linux Programming Interface

#

I've been putting this off for like, 5 years

paper echo
halcyon trail
#

thanks for the recommendation

paper echo
#

used to see them on freenode #python a lot as well, not sure if they moved to libera

grave jolt
#

there are also standard asyncio utilities for subprocesses with a similar interface

paper echo
#

the reason i like anyio is because it has good support for structured concurrency w/ task groups

#

doing the same in asyncio can be obnoxious

lilac oxide
#

Lets say, stripping all features like templates, views etc from Django and embedding DRF and optimising for fast API creation and deployment. But not losing compatibility with Django modules. An out of the box RESTful API Framework in Python. Any opinions?

modern night
#

hmm, just realised I'm missing a print, will need to look into that

valid rose
lime sky
#

yo maybe not the best place to ask but not really sure where to haha

#

anyone here experienced security ppl that could assist with tracking illegal wildlife traders?

#

basically doxxing them to the police

raven ridge
lime sky
#

maybe thats not the right word..

raven ridge
#

it sounds like something that could just as easily be used maliciously as not, and we'd have no way of knowing how it's being used.

lime sky
#

its not "exposing" anyone, its simply finding the info and passing it to the police

#

of how whats being used

raven ridge
#

right, the doxxing part is the finding the info in the first place. We won't help with that, because we can't know how the info would be used.

#

And doxxing bad people is still doxxing, and still against the rules.

lime sky
#

the info would be shared with a conservation society in SEA as well as the police there

#

this is not a malicious endeavour, its one to try and tackle the rampant illegal wildlife trade that threatens many primates/other animals with extinction

#

and i told them i would see if i could find someone that may be able to assist

raven ridge
#

Sounds like a noble cause, but we still cannot help.

lime sky
#

i understand the premise is against the server rules and it cant be discussed in here but maybe there is someone here with the knowledge that could assist me privately

mild flax
#

!rule 5
This rule covers any practical discussion of projects that could be used maliciously. We also do not want to facilitate seeking private help for the same reasons.

fallen slateBOT
#
Rules

5. Do not provide or request help on projects that may break laws, breach terms of services, or are malicious or inappropriate.

mild flax
#

This conversation is also off-topic for this channel. If you have further questions regarding our rules please message @summer lichen.

lime sky
#

i understand what youre saying, that the same knowledge can be used maliciously, but thats simply not the case here

#

its also separate from discord

raven ridge
#

!warn 239000912733929482 We have no way of verifying that, and won't be making an exception to the rules. As you've been told, if you want to discuss this further, send a DM to @summer lichen.

fallen slateBOT
#

:incoming_envelope: :ok_hand: applied warning to @lime sky.

lime sky
#

when you say "discuss this further" do you mean about the rule or about the question haha

#

but alright apologies

raven ridge
#

about the rule, and why we won't be bending it.

lime sky
#

yes understood

unkempt rock
#

i have a question

#

i want to make a password vault

#

project that stores all my passwords

#

would i need sqllite3 in order to store my passwords

#

im planning on using tkinter for gui

rich cradle
radiant lily
#

anyone familiar w python webscraping?

native adder
#

Ja, just ask your question

deep bramble
#

Why does __exit__ take 3 parameters: exc_type, exc_inst, exc_tb, when it could easily just work with the exception instance, from which we can get the class jut by doing exc_type.__class__ or the traceback with using traceback stdlib package. Is there a reason why this was done?

halcyon trail
#

I've wondered that before too

peak spoke
#

I believe that's just the usual way to represent an exception

deep bramble
#

exception is represented well enough just by the actual exception instance, I don't see any need for the other params

spark magnet
#

it might be a historical leftover from when strings could be raised, and were not objects.

deep bramble
#

hm, I wonder why that wasn't removed in py3

#

I don't think you could raise strings in first python 3 versions

peak spoke
#

I'd also guess that it predates the traceback module by quite a bit so the traceback was useful.

peak spoke
spark magnet
#

I could be wrong about the reason

halcyon trail
#

The historical explanation occured to me as well. And maybe traceback was added later as a module

#

That could be verified easily

acoustic crater
#

I think someone recently said how to get un-instantiated instance methods from super(), I can't find it

#

cuz super().__class__ gives you the super class

#

is it possible? I'm trying to write a fix for Pillow so I can just use Image.Image but if, for some reason, the inheritance chain gets messed with it'd be nice if it automatically worked anyway

dusty oak
#

Awesome. Finishing reading head first book about Programming Patterns.
This book is indeed... worthy of thousands books. Amount of brain-friendly new... levels of programming philosophies is overwhelming
https://www.amazon.com/Head-First-Design-Patterns-Brain-Friendly/dp/0596007124
https://www.youtube.com/watch?v=hGIW2fDb0jg

Head First Design Patterns: A Brain-Friendly Guide: Eric Freeman, B...

Head First Design Patterns: A Brain-Friendly Guide [Eric Freeman, Bert Bates, Kathy Sierra, Elisabeth Robson] on Amazon.com. FREE shipping on qualifying offers. Head First Design Patterns: A Brain-Friendly Guide

YouTube
Fil Far
Programming ▫️ Designing ▫️ Hacking ▫️ Coding ▫️ Focus ▫️ Music 🧬

Focus & Concentration
(Without Vocals)

📌 SHORT DESCRIPTION
So here we are, one bigger mix. :)
As I said in this mix are only few songs new.. this mix is compilation of the best focus songs. I removed one more aggressive or distracting songs and added some new. So hope you will enjoy it. I also make some really cool visuals for video to have bet...

▶ Play video
acoustic crater
#

that book's about Java...

dusty oak
gleaming rover
#

whether a particular pattern is useful or idiomatic will depend on the language

dusty oak
#

don't you go to in built into python patterns

#

yes, there are super specific to language

gleaming rover
#

that said

dusty oak
#

small correction to consider current language can be done too)

gleaming rover
#

I don’t think the fact that it’s a Java book disqualifies it

dusty oak
#

the brain-friendliness makes it worthy to learn even you are pythonist

#

after all, there is no equivalent book in python about it

rancid osprey
#

oh

gleaming rover
rancid osprey
#

advanced

#

wow

#

can i be advanced

gleaming rover
rancid osprey
#

print("im advanced".format(EWFSRGERE)")

#

oh mb

sand goblet
#

I wonder why it wasn’t named “implementation-discussion” or something like that

spice pecan
#

We've iterated through some names, advanced discussion seems to be the best option so we've sticked to it

#

It's not just about the implementation, it's also about advanced concepts, such as metaclasses, for example, so it's difficult to pick an exact name

kindred rapids
#

dm @summer lichen instead of pinging the admins

raven ridge
#

@unkempt rock As Cookie says, send a DM to @summer lichen if you believe there's some action that the staff should take. Don't bring up personal grievances in public channels, and especially not in on-topic channels.

spark magnet
austere spruce
#

@spark magnet let's hear the hot take.
As I see it, if you need to make sure that a class only has one instance and provide a global access point to that instance Singleton is the right way to go.

flat gazelle
#

consider that modules already do that exact thing

spark magnet
#
  1. it's easier by far to simply make one instance and access that global
prime estuary
#

And in many cases you could just use the module itself and some functions.

spark magnet
#
  1. classes that can only be instantiated once make it very hard to write isolated unit tests, so people build yet more hackery to get around the singleton-ness
wide shuttle
#

Some design patterns just deal with limitations of the languages in which they evolved

#

I don't really buy the "universality" principle of all design principles

austere spruce
#

@spark magnet
Thanks for breaking your points down

  1. Let's try to narrow the scope a bit;
    Let's say that we want to implement a RESTful API.
    We have a few internal requirements:
    • It's going to talk to a database when it's running, but we want to be able to decouple thay layer with an in memory data structure to have fast unit tests while developing.
    • We want to keep our "routes" very thin, and instead have the logic in some domain service class. This class is then instansiated with a SomeDomainNameQueries class that can handle the different data queries we want to be able to handle (and that class then depends on the actual data model (fx. an ORM).

We don't want to instantiate this Service class in every route where we need it with a concrete SomeDomainNameQueries class, so we instead make a service container.
This service container class is then responsible for returning a concrete instance of the Service class that the different routes can use via dependency injection.

Now this concrete service class, we only really want one instance of that, so here to me a singleton pattern makes sense.

We could then in our test mock the ServiceContainer class to create the different instances of the services with a SomeDomainNameQueries class that could implement the different query operations and save them to a dict or w/e datastructe that makes sense.

spark magnet
#

i'm wondering about a lot of this design. Why do you want to have only one instance of the concrete service class?

austere spruce
grave jolt
#

I think 'the gang of four' doesn't buy into universality either

unknown.png
#

This class is then instantiated with a SomeDomainNameQueries class that can handle the different data queries we want to be able to handle
well, then you can just pass the SomeDomainNameQueries as an argument, right?

austere spruce
austere spruce
spark magnet
#

@austere spruce you say "i don't need that", but what's the harm in instantiating the concrete service class many times? Do you need it to be a singleton? Why enforce that?

austere spruce
spark magnet
#

i was afraid you were going to say faster 🙂

austere spruce
#

haha

grave jolt
#

creating a global variable will give even faster access, and it doesn't hide your intent of making a global variable

austere spruce
#

to me it was just the: "As soon as I have create one of these things, I always want to create them with the same arguments".

#

so I only really need one, so why not just make it a singleton.

spark magnet
#

or, make it the right way each time you need it.

austere spruce
#

Maybe I am reading into this wrong, but it seems like you don't like the singleton pattern? Or is it merely "I don't think it's a clear way of doing things, and you can do it in these other ways which I think is more clear"

austere spruce
grave jolt
#

You can also expect an object of some shape, like in classic dependency injection. For example (pseudocode):

class InMemoryDatabase:
    def __init__(self, pictures):
        self._pictures = pictures

    def find_picture(self, id):
        return self._pictures.get(id)


class PostgresDatabase:
    def __init__(self, connection):
        self._pool = PostgresPool(connection)

    def find_picture(self, id):
        with self._pool.acquire() as conn:
            cursor = conn.execute("SELECT (name, url) from PICTURES WHERE id=%s", (id,))
            row = cursor.fecthone()
        if row is None:
            return None
        return Picture(id=id, name=row["name"], url=row["url"])


def make_routes(db):
    return <something>

and then pass the 'production' instance at your entry point and pass the 'testing' instance in your tests

#

but I know that Python developers are usually allergic to this type of thing

spark magnet
austere spruce
#

yeah that's what I am doing in my tests normally :-)

albeit I have an extra layer for this SomeNameQueries so I don't need to implement all ORM methods

spark magnet
#

(and btw, sometimes they aren't even problems)

austere spruce
#

@grave jolt
I would then have

class DatabaseUserQueries(UserQueries):
   ...

and then In my test:

class TestUserQueries(UserQueries):
    ...
austere spruce
spark magnet
#

@austere spruce sometimes what's going on is that people know that globals are bad, so they don't like the idea of putting this into a module: the_thing = Thing(). But a singleton is a global, just dressed up in fancy clothes.

austere spruce
spark magnet
#

and if you find you need your tests to do something tricky like patch, that's a clear sign that the class wasn't by its nature a singleton.

austere spruce
grave jolt
#

An example of a global variable that's not necessarily a singleton is asyncio's event loop. Sure, at any single point there can only be one event loop running. But you can set a different one (like uvloop), or stop and garbage-collect the current one and create a new one

austere spruce
austere spruce
#

In any case, time to go outside and enjoy the weather, thanks for the discussion! Have a good one

grave jolt
#

sounds terrible

unknown.png
austere spruce
# grave jolt sounds terrible

Yeah it was 24C at 8 when I woke up, absolute hell. I went to where I work to stay there instead since they have AC kekLeo

lunar trail
#

@grave jolt what does f"{x!s}" do?

grave jolt
lunar trail
#

But surely that doesn't join them?

grave jolt
#

actually, it's already faster with just an f-string, but !s reduces a little bit of overhead because __format__ accepts two arguments

grave jolt
lunar trail
#

It won't be like 0, 1, 4, 9, 16

grave jolt
#

You can do

", ".join([f"{x!s}" for x in xs])
lunar trail
#

!e py xs = [0,1,4,9,16] print(", ".join(f"{x!s}" for x in xs))

grave jolt
#

!e

xs = [0,1,4,9,16]
print(", ".join([f"{x!s}" for x in xs]))
fallen slateBOT
#

@grave jolt :white_check_mark: Your eval job has completed with return code 0.

0, 1, 4, 9, 16
#

@lunar trail :white_check_mark: Your eval job has completed with return code 0.

0, 1, 4, 9, 16
grave jolt
#

f"{x!s}" it does the same thing as str(x), but faster 🙂

lunar trail
#

I don't see how making it a list is quicker

#

Because you're doing more work

grave jolt
lunar trail
#

Huh

grave jolt
#

I was surprised as well 🤷

lunar trail
#

Seems like a weird decision but fair enough

#

What about list(gen) vs [x for x in gen]

#

list(f"{x!s}" for x in ...)

grave jolt
#

I'd expect list(gen) to be slower

grave jolt
fallen slateBOT
#

@grave jolt :x: Your eval job has completed with return code 1.

001 | sending not strings:
002 | sending 1
003 | sending 2
004 | sending 3
005 | sending 4
006 | Traceback (most recent call last):
007 |   File "<string>", line 6, in <module>
008 | TypeError: sequence item 1: expected str instance, int found
grave jolt
#

first the generator runs until the end, and only then join checks if it resulted in strings.

native flame
#

doesn't f"{x}" call .__str__ by default?

grave jolt
#

!e

class Foo:
    def __format__(self, format_string):
        return "quack, but also " + format_string

    def __str__(self):
        return "meow"

print(f"{Foo()}")
print(f"{Foo():aaa}")
fallen slateBOT
#

@grave jolt :white_check_mark: Your eval job has completed with return code 0.

001 | quack, but also 
002 | quack, but also aaa
grave jolt
#

When an object doesn't have a __format__ method, {x} falls back to __str__, and if it doesn't have a __str__, it falls back to __repr__.

native flame
sturdy timber
# grave jolt That's because `join` in CPython first makes it into an array of objects anyway.

I looked into this, and remember reading that it was done as the iterator needed to be iterated over twice (so it could work out how much space to allocate for the string or something, idk), although I don't think that would explain why the same thing would apply to things like sum, i.e. sum(i for i in range(10000)) is slower than sum([i for i in range(10000)]), I tried to look at the source code and I couldn't see anywhere that the generator was being turned into an array (although granted I didn't really know what I was looking at). I sort of came to the conclusion that generators were just... slower, because they're python constructs rather than c constructs (for lack of a better way of saying that, because I don't really know what I mean). Any ideas if i'm misunderstanding something, or am I along the right lines...

grave jolt
#

it could work out how much space to allocate for the string
Well, CPython doesn't seem to do that, because it first exhausts the iterator, and only then checks if any of the elements are not strings

#

With sum it doesn't seem all that clear, it depends on the input size.

unknown.png
sturdy timber
#

oh, hm, yeah, list comp only seems to be faster for small input sizes from what I can tell. I guess that's just because the list needs to be reallocated as it grows, although with small inputs there's a fairly significant difference, so I guess that could be due to generators being slightly slower to iterate over, which makes sense.

sturdy timber
peak spoke
#

You could just be seeing the generator iteration overhead on lower input sizes for the sum

grave jolt
#

Yeah, maybe the allocation time starts being an issue only in larger list sizes.

fallen slateBOT
#

Hey @static bluff!

Uh-oh! It looks like your message got zapped by our spam filter. We currently don't allow .txt attachments, so here are some tips to help you travel safely:

• If you attempted to send a message longer than 2000 characters, try shortening your message to fit within the character limit or use a pasting service (see below)

• If you tried to show someone your code, you can use codeblocks
(run !code-blocks in #bot-commands for more information) or use a pasting service like:

https://paste.pythondiscord.com

static bluff
#

I know its hard to tell without context, but does this make sense?

#

Feel free to ask any questions, of course

paper echo
#

Tbh it only makes any sense at all because I know your weird naming conventions

#

(Joking)

#

For real, the notes and docstrings are VERY helpful

#

The if not inside the while loop is weird

#

That might never trigger?

static bluff
#

😄