#internals-and-peps

1 messages · Page 118 of 1

spark magnet
#

I think this is one of the risks of python typing: people will think only about what they are passing, not what they could pass.

magic python
#

well afaik all hte time it's used its passed with an index, just that none of the methods available to pd.Index are used. A similar example - if something takes a pd.Series, but uses none of the pd.Series methods (just treats it at a list), then is it better practice to have the function take the most simple type (list) possible?

spark magnet
#

again, Iterable is probably what you want.

flat gazelle
magic python
halcyon trail
#

It really depends, it's a trade-off

visual shadow
#

This is why we can't have nice things. Because ducks are not types for quacks sake

flat gazelle
halcyon trail
#

you can either be more lenient in what you take (take the most minimal type possible), or be more restrictive in what you take

magic python
halcyon trail
#

If you're more lenient you make things easier for users, but if you decide to change your function implementation at some point, there's more of a chance you may want to have access to those other methods, which would require changing the type annotation, and having a breaking change

#

It just depends

magic python
#

depends by project or case? Is this not something people pick one approach with and stick to?

halcyon trail
#

I wouldn't say it's more common to use Iterable[int] than list[int] necessarily, it depends what you are doing

flat gazelle
#

ye, you have to decide on a case by case basis

halcyon trail
#

no, it depends function by function

magic python
halcyon trail
#

however, Sequence[int] should probably be more common than List[int] for function arguments

#

that's what you see people in python often recommend, but I think they're not fully understanding the trade-offs tbh

flat gazelle
magic python
#

infinite iterable being a generator?

halcyon trail
#

no

magic python
#

oh 🤔

flat gazelle
#

infinite iterable is an iterable which never raises StopIteration

#

generators are a different thing, and are also iterables

#

but you can make an infinite iterable which is not a generator

#

e.g. itertools.count()

halcyon trail
#

that's a reasonable point I guess but I still think that function is typed fine as written

#

it's a trade-off

#

I suppose a decent middle ground would probably be something like Collection[int]

#

Collection is something that suports size, iteration, and contains

magic python
#

i don't really know why i'd choose iterable over seq or visa versa

halcyon trail
#

Sequence is a non-mutable List, basically

magic python
#

Use Iterable for generic iterables (anything usable in "for"),

and Sequence where a sequence (supporting "len" and "getitem") is

required

from the doc

halcyon trail
#

So, Sequence API includes things like length, and random access

#

if you need random access or length, you use Sequence

#

if you are just iterating over it, then Iterable is sufficient

#

if you want to guarantee that it's a finite Iterable, then you could use Collection, although it doesn't seem like that's encouraged

#

it's tricky I guess because there are valid iterables that are finite, but are not Collection

#

as always, types cannot be exactly as precise as you want them to be, Iterable will include some things that are bad and Collection will exclude some things that are good

#

In python it's fairly common to use generators and generator expressions, and I think having false positives is worse than false negatives, so probably I'd use Iterable even in cases where it shouldn't be infinite

magic python
#

cool, i think i have a lot of list[int] to replace 😅

flat gazelle
#

SOLID will tell you to always use the narrowest interface you can, but well, that doesn't really work in software that needs to be updated

halcyon trail
#

FWIW I've used a ton of List and Dict all over my codebase (incorrectly)

#

well, if SOLID tells you that, SOLID is wrong 🙂

flat gazelle
#

yup

halcyon trail
#

if you're pretty confident you can live with the narrower interface, live with the narrower interface

#

but, if you aren't, then you should do exactly the opposite: stick with the wider interface

#

because you can always narrow the interface in the future, upon further reflection, or upon request, as it's not a breaking change

magic python
#

so minimise the chance of breaking changes, i guess

halcyon trail
#

accepting the narrowest interface, that is. This gets a bit confusing 🙂 A function that accepts a narrower interface has a wider API, and a function that accepts a wider interface has a narrow API

magic python
#

which (to me) felt a little counterintuitive, as i thought mypy was meant to break everything lol and make it super strict

halcyon trail
#

mypy is meant to enforce things for you, to the degree that you indicate that you need them

#

If you're not sure whether your function needs Sequence or Iterable, and you really don' tknow how the function will evolve, use Sequence

#

You can change it to Iterable in the future easily

#

However, if you are quite sure Iterable is sufficient, then stick with Iterable, there's no point forcing users to pass in Sequence if it's just unnecessary

flat gazelle
#

one merit of iterable is that you can then do f(x for x in y if z), which is not possible with a Sequence, but alas, not a big loss compared to having to deal with changing the public API in the future

magic python
#

i find that mypy makes me use less comprehensions, stuff like l = [re.match(exp, s).group(1) for s in m] will give me Item "None" of "Optional[Match[Any]]" has no attribute "group, so I convert these to for loops and put a raise or whatever in the case of the match being None, idk what's best really tho

halcyon trail
#

it's an engineering decision:

  • how much headache does it cause for users to have a narrower API
  • how quickly can you widen the API for users, if the narrow API is an issue
  • how likely is your implementation to change in such a way that you want the narrower API
#

etc

#

@magic python you can use a list comprehension there still with a small helper function

# 
T = TypeVar("T")

def unwrap(x: Optional[T]) -> T:
    if x is None:
        raise RuntimeError("Unwrapped empty optional!")
    return x
#

l = [unwrap(re.match(exp, s).group(1)) for s in m]

magic python
#

i haven't understood these T yet 🤔 but ok cool that's interesting

halcyon trail
#

a lot of other languages have syntactic sugar for this

#

T just means its generic

#

unwrap can accept an Optional[T], for any T

#

and it returns a T

magic python
#

why use T instead of Any

halcyon trail
#

What would it return if you use Any?

magic python
#

I guess Any again 🤔

halcyon trail
#

Right. That's why you use T.

magic python
#

so T does type inference?

halcyon trail
#

Yeah, the T is inferred, and then can be used in other places

magic python
#

hm ok, cool

halcyon trail
#

so if you pass unwrap an Optional[int], mypy understands that it returns an int

#

which is important

terse orchid
visual shadow
#

Oh it was just a tongue in cheek comment related to duck typing

magic python
halcyon trail
#

what do you mean exactly?

magic python
# halcyon trail what do you mean exactly?

This:

import re
from typing import Optional, TypeVar

import pandas as pd

T = TypeVar('T')

s = ['one this is one here\n[two]', 'three this is three here [five]']

def unwrap(x: Optional[T]) -> T:
    if x is None:
        raise RuntimeError("Unwrapped empty optional!")
    return x

# a
df_index = (
    pd.Series(s)
    .str.replace("\n", " ", regex=False)
    .apply(lambda x: unwrap(re.search(r"^(.*) \[", x).group(1)   ) )
    .str.wrap(4)
)

gives:

check.py:21: error: Item "None" of "Optional[Match[Any]]" has no attribute "group"  [union-attr]
#

the error still occurs for regular comp actually:

k = [
    unwrap(re.search(r"^(.*) \[", x).group(1)   ) for x in s
]

throws the same error as above

halcyon trail
#

that's because the unwrap call is happening in the wrong place

#

I didn't look carefully enough at the original error

magic python
#

oh it shouldn't wrap .group

halcyon trail
#

but you should get used to reading these errors and diagnosing them, if you want to use types. In this case, it's telling you that the thing you're calling .group on is an optional, right

#

right

#

you need to unwrap before you call group

magic python
#

is unwrap a convention for this sort of thing?

halcyon trail
#

in some languages

#

Rust uses unwrap

magic python
#

i mean, is this "unwrapping", hm ok cool

halcyon trail
#

well, unwrapping is the noun, unwrap is the verb, and shorter, so better as a function name

#

A lot of languages don't have names for it because they have syntactic sugar

magic python
#

right, just id have used "raise_if_none" or something i guess as i've never heard of this

halcyon trail
#

Kotlin for example would just do re.search(...)!!.group(1)

#

actually seems like python's none-aware operator proposal doesn't even cover this form anyway

magic python
#

typing is annoying sometimes but it has kinda forced me to think about what is actually going on a bit, which is probably healthy

halcyon trail
#

yeah

#

I mean Optional is one of the most classic examples, because even some strongly typed languages tend to ignore it in their type systems

#

like Java most notably, and C/C++ in the context of pointers, which is pretty bad in C (but not as bad in C++, which has references)

#

Without mypy, you of course would just be calling .group on something that could be None, and it would blow up

#

mypy forces you to be explicit, in this case with unwrap it's still a not-terribly informative exception ultimately but at least people readin gthe code can see the explicit unwrap

#

You could also just define a local function similar to unwrap but with an exception message tailored to what's going on

lethal magnet
#

is there a place anywhere where i can send a zip and get it analyzed?

#

for viruses

cloud crypt
flat gazelle
paper echo
#

@magic python @halcyon trail

import re
from typing import Optional

import pandas as pd

def search(x: str) -> Optional[str]:
    if m := re.search(r"^(.*) \[", x) is not None:
        result = m.group(1)
    else:
        result = None
    return result

s = ['one this is one here\n[two]', 'three this is three here [five]']

df_index = (
    pd.Series(s)
    .str.replace("\n", " ", regex=False)
    .apply(search)
    .str.wrap(4)
)
#

the "unwrapping" is kind of a red herring

#

i have a feeling you're going to run into the same problem you'd have with monad transformers, where there's no "general" way to unwrap a value that's in a monadic context

#

stated another way: you have to actually do the unwrapping in order to unwrap the thing

#

in a functional language (or python 3.10) you'd pattern match, in python you use if, and mypy knows the if x is None pattern and will refine the type of x accordingly

#

sometimes you have to refactor your code to make the types more amenable to analysis - 90% of the time this is a good thing and either forces you into better patterns or at least doesn't make your code worse

#

the other 10% of the time is when you're interacting with untyped code and/or untyped or recursively-typed data (which mypy can't handle yet)

halcyon trail
#

what do you mean by a "red herring" ?

#

@paper echo

#

it's just a different approach

#

in the original code, they wanted to throw if something was None

paper echo
#

oh, i missed that

#

still, i think trying to "generically" unwrap is an awkward abstraction in python

halcyon trail
#

what's wrong with that unwrap function?

#

seems fine to me

paper echo
#

nothing, but i think it's an awkward abstraction

halcyon trail
#

I'm just not sure what's awkward about it 🙂 Most languages (at least, those that aren't pure FP) hav esomething like this

#

Rust has unwrap, C++ optional has .value, Kotlin has !!

paper echo
#

i guess i'd rather write that search function - the type unwrapping stuff falls out naturally from the logic

#

but isn't !! null-propagating?

halcyon trail
#

!! throws on null

#

it's exactly unwrap

paper echo
#

ah

halcyon trail
#

the point is that most languages that have an optional type try to let you do both

#

having to write that search thing by hand is pretty awful

paper echo
#

fair enough, but that's because python has doodoo lambdas

#

i see the value, i guess i feel like "throw on null" is such a specific and uncommon thing to want and i'm surprised it's baked into a lot of languages

halcyon trail
#

Yeah, I'd probably still prefer something taking a lambda in that case, usually you'll just be calling a pre-existing function perhaps with some arguments locked down, which isn't completely awful with python lambdas

paper echo
#

propagating null is a lot more useful imo

halcyon trail
#

it's not specific and uncommon at all

#

it's actually extremely common that based on other parts of the code, or your knowledge of the input, that you know that nulls are not possible, or represent a grave error, and you simply want to throw

#

Actually, an unwrap with optional msg and exception_type arguments would be extremely useful in python and cover a lot of situations

#

Propagation is probably more useful overall, at least in Kotlin I found myself using propagation more, even though Kotlin supports both conveniently, and uses exceptions still quite a bit

#

But partly that's because, in the first place, Kotlin functions tend to throw for grave errors.

#

In Rust where everything is Result, for example, I tend to see quite a lot of unwrap. But I guess taht's not null, so I'm not sure if you'd include that.

# 
def unwrap(x: Optional[T], msg="Unwrapped empty optional!", exception_type=RuntimeError) -> T:
    if x is None:
        raise exception_type(msg)
    return x

def map_optional(x: Optional[T], func: Callable[R, [T]]) -> Optional[R]:
    if x is None:
        return None
    return func(x)

k1 = [
    unwrap(re.search(r"^(.*) \[", x), f"Regex did not match {x}!").group(1) for x in s
]

k2 = [
    map_optional(re.search(r"^(.*) \[", x), lambda x: x.group(1)) for x in s
]

Neither is super beautiful but it's the best we can do I guess

pseudo cradle
#

That's interesting because I made an update to production code to propogate nulls in cases where computations were unreliable, and it wound up breaking a lot of things. I'm brainstorming ways to handle that, but I am not sure what I can do other than maybe adding a decorator that checks if any of the positional arguments are null.

#

Otherwise I need if statements/try-except clauses everywhere which just seems bad

unkempt rock
#

this isn't a code specific question

#

but a project question.

#

can i use sqlite3 to create a database that stores scores from throughout marking periods for gpa purposes can i create a table using the user input that i get from within the program?

pseudo cradle
#

Not sure if this is the channel for this, this is more of advanced discussion regarding the language itself. However, I don't see why you wouldn't be able to do what you just described.

unkempt rock
#

ok thanks

static bluff
#

What are we discussing today?

spice pecan
#

None-handling

#

So null propagation, unwrapping Optional, Result<T, E>, etc

halcyon trail
#

@pseudo cradle well, i guess it really depends what happens before you propagated the null, what exactly "unreliable" means

#

in many cases but not all breaking loudly is preferable to brekaing quietly

#

as to what can be done, well, mypy will handle this for you, right

#

if you have an argument annotated as int and you have another function that returns Optional[int], you won't be able to call that function and pass it directly to the argument

pseudo cradle
#

I don't think I've ever used Optional before

halcyon trail
#

So in the mypy world, as soon as you change a return type to Optional, it should be able to find all the breaks via static analysis, unless you are doing fairly fancy stuff

pseudo cradle
#

Hmm

halcyon trail
#

do you use mypy?

pseudo cradle
#

No. I probably should

grave jolt
#

or pyright

pseudo cradle
#

I'm a digital signal processing engineer, while I try to filter messy signals sometimes I have to remove them altogether if they're bad enough. I added a decorator that makes it so if there's not enough good signal for a calculation, I just return None. But sometimes I call those functions within other functions, that's where it's breaking and I am not sure how to handle it.

flat gazelle
#

why not raise an exception and catch it at a step where it makes sense to try the next signal

pseudo cradle
#

Mostly because it would be a lot of try and excepts, which I feel like would clutter our pipelines.

#

I'm trying to avoid that, if possible.

#

Ideally there would be a way to have a try and except clause for each line in a function without having to manually write all that out.

#

I tried replacing the bad data with numpy NaNs and then use nan related functions, and that sort of works but has its own headaches

static bluff
#

Suppress?

#

No context here, but the contextlib_ext module has a decorator that tries a function and returns Nome if it catches an error

pseudo cradle
#

Hmm

#

That is an option. I'll make a note of it, thank you!

static bluff
#

I wish it had a fallback decorator though. Try this, if it fails, do that

pseudo cradle
#

That shouldn't be too hard to set up yourself though, right?

halcyon trail
#

but what would suppressing buy you, that would be the same as not signalling the error at all

#

unfortunately there probably is no real silver bullet, handling errors is hard, especially if you have many points of failure.
Pick your poison based on what makes sense. Throwing can be pretty good because you can do many operations without constantly looking for optional/nan, and catch at a slightly higher level

flat gazelle
#

^ the ideal thing here would probably be conditions, you could just set up a restart that would pick the next signal instead, but I haven't seen a good impl in python yet

halcyon trail
#

can you explain what you mean by that?

flat gazelle
#

are you familiar with conditions as a form of error handling, afaik the only language that actually implements them rn is common lisp, but they were in very early versions of rust, but got removed.

halcyon trail
#

no

flat gazelle
#

apologies for the delay, went for dinner.
The idea of a condition is that rather than working like an exception and unwrapping the stack, it instead calls a so called "restart" immediately (essentially a callback), which then decide what to do. For example, you can set things up such that a NameError would use some default fallback (not a great idea) instead of truly erroring out. By default, there are 2 restarts - act like an exception and kill the program, but you can add more at runtime dynamically. You can always do things a condition can do without a condition, but conditions are just kind of convenient. You can even unwrap the stack partially and do something at some different part of the call tree.

#

the problems are obviously complexity and not being checked at compile time, which is why rust went with result types instead

paper echo
#

afaik a condition is pretty much a delimited continuation

flat gazelle
#

pretty much

#

but continuations are rarely the correct tool, since they are far too flexible

paper echo
#

(or at least, they're conceptually similar, even if they're not technically continuations under the hood)

flat gazelle
#

for example, scheme cannot have the equivalent of a context manager that will always call close, like unwrap-protect of CL

paper echo
#

i tried to implement something like this with generators, but as soon as you wanted to actually use generators in your code, it kind of blew up

flat gazelle
#

exactly due to continuations

paper echo
#

lisp people love the lisp condition system

#

i'm with them, i feel like it's the right balance of power and usefulness

flat gazelle
#

yeah, I understand why rust decided against it, but it is a nice system

paper echo
#

maybe it's too much for a "systems" language

#

but i would love it in python

flat gazelle
#

same

#

though the simplicity of exceptions maybe has its place more

#

even if they are sometimes unwieldy to handle

paper echo
#

can't you just ignore the restart functionality?

#

but i guess that means more complicated "professional" code

flat gazelle
#

I feel like a large convenience of conditions is being able to test them out interactively in the REPL, which is somewhat incompatible with the idea of ignoring restarts by default

halcyon trail
#

I feel like I'd need to see an example to really see where the benefit lies

#

does this callback have to be passed explicitly into every function, is it a global thing that you can temporarily change in a certain context, how does it apply to functions getting called transitively, etc

paper echo
#

imagine you could do this @halcyon trail :

def f(x):
    try:
        print(x[5])
    except IndexError as exc:
        exc.restart(None)

assert f('hello world') == 'o'
assert f('') == None
#

Example from https://gigamonkeys.com/book/beyond-exception-handling-conditions-and-restarts.html with my docstrings added:

(defun well-formed-log-entry-p (text)
  "Is the log entry well-formed?"
  ...)

(define-condition malformed-log-entry-error (error)
  "Condition signaling that a log entry is malformed."
  ((text :initarg :text :reader text)))

(defun parse-log-entry (text)
  "Parse a single log entry. This should be a single line of text."
  (if (well-formed-log-entry-p text)
    (make-instance 'log-entry ...)
    (error 'malformed-log-entry-error :text text)))

(defun parse-log-file (file)
  "Parse a log file, with one log entry per line."
  (with-open-file (in file :direction :input)
    (loop for text = (read-line in nil nil) while text
       for entry = (handler-case (parse-log-entry text)
                     (malformed-log-entry-error () nil))
       when entry collect it)))

the python equivalent (using the hypothetical exc.restart) would be something like

def is_well_formed_log_entry(text):
    """Is the log entry well-formed?"""
    ...

class MalformedLogEntry(Exception):
    """Condition signaling that a log entry is malformed."""
    def __init__(self, text, *args):
        # I don't know how to do this "properly" for custom exceptions
        self.text = text
        super().__init__(text, *args)

def parse_log_entry(text):
    """Parse a single log entry. This should be a single line of text."""
    if is_well_formed_log_entry(text):
        return LogEntry(text):
    raise MalformedLogEntry(text)

def parse_log_file(file):
    """Parse a log file, with one log entry per line."""
    entries = []
    with open(file) as fp:
        for text in fp:
            try:
                entry = parse_log_entry(text)
            except MalformedLogEntry as exc:
                exc.restart(None)
            if entry:
                entries.append(entry)
    return entries
halcyon trail
#

what does exc.restart do?

#

Okay, I think I understand what this lets you achieve over exceptions/results

#

whether it's actually a good idea to let you have that much flexibility seems more questionable (to me)

#

the problem is that once this becomes your error handling strategy, you basically have inversion of control, everywhere, implicitly

paper echo
#

what do you mean everywhere?

#

at every exception, yes

#

or more generally, at every signalled condition

#

but the point is that your code now explicitly doesn't know what to do anymore, and it's asking for help from "higher up"

#

in python, the only thing you can do is unwind the stack and exit

#

when you add types into this mix, you get the "algebraic effects" i was going on about a while ago

#

i mean yes, theoretically, you can write code with a restart that causes the original code to summon the dreaded nasal demons or launch nuclear missiles, but you can do that with except anyway

halcyon trail
#

except doesn't let you inject code downward

#

the exception is thrown upward

#

if you want some kind of handling on errors, you need your function to take an argument explicitly (or use a global, which we know is bad, mkay)

#

my very personal opinion about where this will end up, I think, is towards more static approaches towards error handling (similar to Result), but properly supported by tooling. I haven't seen any language really come close to this, but if I had to make a guess for the distant future, that would be it.

#

the main complaint against Result like types is that in some cases it can make bubbling up a newly added error tedious, but an IDE ought to be able to do such a thing automatically

paper echo
#

i guess i'm not sure what the problem with injecting "downward" is

#

if someone goofs up and injects garbage into your function, that's on them

#

maybe your concern is that it effectively expands the "api surface" of a function?

#

i've said this before, but i have a theory that common lisp gets away with these ridiculous features because common lisp programmers are unusually disciplined

#

(other than the let-over-lambda people)

halcyon trail
#

idk, I think CL gets away with it because lisp is almost never used in large scale projects

#

most people use CL on hobby projects or on their own, and love it. I have seen horror stories from people who worked with CL at companies and had to deal with the worst kind of code

#

other peoples' code 🙂

#

The problem with injecting downward for starters is that it violates referential transparency

#

(unless you inject downward explicitly, via a function argument)

magic python
#

Causes mypy to generate a text file report documenting the functions and lines that are typed and untyped within your codebase.

from : https://mypy.readthedocs.io/en/stable/command_line.html#cmdoption-mypy-linecount-report

Which gives output like:

   1286   13487    368    866 total
    286     386     13     14 something.somewhere
...

there are no headers in the file though for the columns, so presumably for module something.somewhere we have 286 lines typed, 386 lines which aren't typed, 13 functions which are typed, and 14 functions which are untyped?

#

it's also not too clear what the definition of "precision" is wrt mypy 🤔

raven ridge
#

My guess would have been that it's 286 lines out of 386 are typed, and 13 out of 14 functions

magic python
paper echo
past ingot
#

just a simple question
flutter or react-native
im getting into mobile development and im fairly new to dart, whereas im fairly experience in js but I want to work with regard to the future so what should I learn?

paper echo
#

!ot @past ingot this channel is about the python language itself, as per the channel description. I think your question is better in an off-topic channel.

fallen slateBOT
unkempt rock
#

Anyone happen to know when __floor__ and __ceil__ were added to floats? I know it was after 3.6.8 rooHmm

#

Apparently in 3.9 OhIPanda

boreal umbra
#

@unkempt rock when are those used?

unkempt rock
#

with math.floor and math.ceil

static bluff
#

Whats with all the people posting assignment questions? Its July, isn't it break for most schools?

tidal marten
#

How do you make a number from two integers where the second integer is supposed to be the fractional part of the number?

wise ibex
#

divide it by 100

#

or 10

tidal marten
#

Is there a more elegant method than a + (b / 10 ** math.ceil(math.log10(b)))?

#

Or should I just do float(f'{a}.{b}')?

tidal marten
grave jolt
#

Why do you need to do this operation?

#

seems like the ceil + log is slightly faster, but IMO the formatting way is easier to understand

unknown.png
tidal marten
#

Yeah, hmm...

tidal marten
#

The ceil + log is faster but less consistent with the time taken?

grave jolt
#

or just capture a separate group for the whole literal

tidal marten
#

There's some clutter as well so I can't just get it

#

Some additional prefix and suffix

deft pagoda
#

if you had the string of b from a regex, calculating the log base 10 would be a lot easier!

tidal marten
#

Hmm, yeah

#

The float method is actually faster tho, if you start with string instead of integer

naive adder
#

is it possible to build a calculator out of string
by i mean out of string i dont mean eval() or exec()
wouldn't it be a lot larger than the original int max value?

visual shadow
#

Could you elaborate on your question? What do you mean

tidal marten
#

float(f'-{number}') seems to be slower than -float(number)

#

Probably because of the string allocation/formatting

deft pagoda
#

i think negating a float is just a bit-flip

#

maybe?

visual shadow
#

Yes. Just the sign bit

tidal marten
#

Does Python do copy-on-write for strings?

#

Like if I want to reference a huge chunk of string in many places, would Python duplicate that exact string or just reference it?

limpid forum
#

everything is passed by reference

#

there are no modifiers to arguments like in other languages because it's all objects so they're treated the same (as opposed to e.g. C# where it depends on whether we have object or basic type) - as references
that's why we can mutate arguments inside functions

livid dove
#

(It ain’t just the syntax that makes python easy for beginners)

flat gazelle
tidal marten
#

Copied when modified right?

flat gazelle
#

strings are immutable, so probably yes

safe hedge
# limpid forum everything is passed by reference

Isn't it technically "pass by object-reference" rather than strictly "pass by reference"? I've read some pieces on this but I have to admit my CS fundamentals aren't really strong enough to know how important that distinction is

flat gazelle
#

yes, it is not pass-by-reference in the C++ sense

#

you can't redirect the reference

sacred tinsel
#

the reference is passed by value

#

😬

charred pilot
#

i've heard "pass-by-assignment"

flat gazelle
#

I have heard pass by value reference as well

#

it's often better to just explain the semantics than to try and come up with the perfect name

charred pilot
#

yeah definitely, it's much more detailed than just a simple phrase

sacred tinsel
#

the terminology makes it tempting to find the corresponding equivalent in cpp but there isn't one as far as I'm aware

safe hedge
#

Is there an example of where it would trip someone up who was expecting it to be traditional "pass-by-reference"?

limpid forum
flat gazelle
safe hedge
#

Because if a=1 in Python a outside the function would be 1 still but in the bottom one it ends up as 2?

sand python
#

but more specifically, a is a whole new name, so changes to a won't affect any names outside the function

flat gazelle
#

the first function would error out with UnboundLocalError

#

oh wait nvm

peak elm
#

but you could always do

def add_one(l):
  l[0] += 1
my_number = 1
tmp = [my_number]
add_one(tmp)
my_number ,= tmp

!

#

(it is gross)

flat gazelle
#

yup, that would work

limpid forum
#

yep, that's how wrappers work - because wrapper's reference doesn't change, only references stored inside the wrapper do

visual shadow
#

Python is pass by assignment. I think the cleanest way to explain it is to just explain it, the lower level abstractions don't easily explain python behaviour

#

If I'm not mistaken, the closest lowest level abstraction would be if the code was doing pass by value, but every variable in the code was a first level pointer..... I would much rather just say pass by assignment and call it a day.

flat gazelle
#

not quite, you can do *ptr = value with a pointer

#

you can't do that with a python reference

visual shadow
#

Yes, you were also not allowed to increase or decrease reference levels

#

Only a first level pointer

spark magnet
#

it's fascinating to me how this idea persists that there are only two choices: pass by value or pass by reference (with people not even agreeing on what pass by reference means).

halcyon trail
#

Yeah, I see people keep correcting each other on what it means

spark magnet
#

well, like lots of descriptions of programming languages, the term gets used slightly differently in different contexts.

#

"reference to a value" or "reference to a variable"?

halcyon trail
#

Yeah. More standardized terms would help

raven ridge
#

!warn 798380170586882109 Your first message on the server being an off topic meme in an on topic channel is not appropriate. Please reread our rules before engaging further.

fallen slateBOT
#

:incoming_envelope: :ok_hand: applied warning to @unkempt rock.

visual shadow
tidal marten
# flat gazelle strings are immutable, so probably yes

I was just thinking whether to pass the actual string or create an object which holds the string and pass the object to reference the string. I'm not manipulating the string outside of getting some slices and whatnots so I don't want multiple copies of the same string.

flat gazelle
#

the slices will be copied out, so if you don't want that, you could make your own string type, but I wouldn't bother

#

python caches strings to save on memory as well

halcyon trail
#

I remember taking it as a given that in GC languages with immutable strings, that substrings wouldn't make copies

#

but then learned that in Java, they actually changed it back from not making copies, to making copies

#

usually though in python I wouldn't worry about something so small, unless your strings are truly huge

#

if your strings are truly huge then you may want a different data structure than the built in str but that's a pretty extreme case

sterile kettle
#

I'm currently developing a simple resnet 50 for a regression which takes in a input image and predicts the target value
but when I try predicting test dataset i always turn out to get the same value
so im not sure, if anyone can point me to some resource that'd be great

azure oar
#

Has anyone ever seen a "MultiException" pattern out there? I was thinking it'd be nice in complex input validations, like CSVs, to have individual exceptions per error, with their own error messages and sets of stackframes, line numbers, and file numbers, but create and aggregate these functions during the course of validation, then raise a combined mega-exception at the end with all the individual exceptions as a collection.

#

Because conceptually, I want to give the end-user all the errors I can find, and not just the first one, so they don't have to upload the same dataset over and over and over with one error at a time fixed. Sure, maybe some of the downstream errors are spurious because of an upstream error, but for the most part at least in my case it'd be valid

halcyon trail
#

I usually just return lists of errors in those cases

azure oar
#

Yeah, I do too

halcyon trail
#

I have dealt with pretty much exactly the seem use cases, i have regression tests, some failures are hard failures and it ends the regression test, but others are not, so I add the failure to a list, and continue

#

and then return the list

#

the most useful thing I can contribute to this kind of situation, is a context manager that takes a list, and pushes any errors into the list

azure oar
#

but I was just thinking "what if it had the whole exception context like a regular exception" so that when an engineer got of an end-user's error, they could trace exactly where in the code that the error occurred and recreate the context, easily, rather than searching for exception messages or something

flat gazelle
#

I seem to remember this in a PEP of sorts

azure oar
#

I could do it with a macro pretty simply

#

Or a pattern that I expect engs to follow

halcyon trail
# 
err_list = list()
...
with append_errors(err_list) as e:
    ...

# If this is one of the errs where you *do* need to early exit
if e:
    return err_list
...
#

what do you mean by a macro?

azure oar
#

adding line and file to the error creation

#

Oh, there are a few packages out there for making macros. none are standard and I'd rather avoid them

#

but I know it's possible

halcyon trail
#

I didn't think you needed a macro to add line and file

#

but at any rate, in the example above, the code inside the context manager can just throw an exception as usual

#

and the context manager can easily save all the traceback information for every error

azure oar
#

Well, you don't. but if I just do errors.append(ExceptionClass("etc etc etc"))) I'm not going to know where ExceptionClass got constructed and "thrown"

#

yeah, okay, you're right.

#

I didn't think of wrapping that in the context manager

#

In my head that'd add a level of indirection but it really doesn't have to, does it...

halcyon trail
#

yeah, it's pretty weird but context managers are a pretty good tool for handling error management related things in python

azure oar
#

thanks!

halcyon trail
#

things probably shouldn't be that way but they are
sure, np. what do you mean by a level of indirection?

azure oar
#

oh, i'd want to pop the first frame off the stack when saving that info, because the first frame would be "in the context manager"

#

I dunno. Until I actually sit down and write it, it's kind of sitting in a mashed up jumble in my head. I may be wrong.

#

But I'm on the right track now, so thanks

#

Oh wait, yeah. I think I get it now.

raven ridge
#

!pep 654

fallen slateBOT
#
**PEP 654 - Exception Groups and except***

Link

Status

Draft

Created

22-Feb-2021

Type

Standards Track

raven ridge
halcyon trail
#

@azure oar It wouldn't be in the context manager

#

you'd throw an exception from some code, it gets caught in teh context manager, the traceback of the exception starts where it was thrown from, not in the context manager

raven ridge
# halcyon trail I remember taking it as a given that in GC languages with immutable strings, tha...

In CPython, it needs to make a new object, if only to store a new pointer into the original string and a length for the slice. That new object needs its own reference count. Unless the substring is huge, I'm guessing that the thought is that the memory allocation is already pretty slow, and the cost of copying data out of the original string is low compared to the memory allocation, so it's not worth optimizing.

halcyon trail
#

Sure, a new object would need to be made in every language, hard to see how that's avoidable

#

I think it's not just that, another issue is "memory leaks", if you have a big string allocated that way, and now you have all these substrings pointing into it, then you need to keep the big string around

azure oar
#

Yeah I figured it out... Thanks y'all

halcyon trail
#

Another thing is that it just simplifies the implementation, which saves you perf in general. Anyhow I just thought it was interesting, it does say something that Java did avoid the copy for a long time, and then switched, and obviously a lot of thought/man hours went into that, so I think it's not super obvious which is better, perhaps

#

np

raven ridge
#

The biggest issue with having slices reference the original string in Python would probably be that the string caches its own UTF-8 representation, and so that would need to be referenced as well - and that's not trivial, since it's multi-byte

#

Though I suppose you could reference the buffer of Unicode codepoints and not the buffer of UTF-8 bytes, and just lazily populate the latter instead

#

Though implementation complexity would definitely be a big consideration as well. The implementation is already much more complicated than you'd naively think.

#

Between caching the UTF-8 representation, and storing the array of code points in different widths depending on whether all of the characters are in ASCII or in the BMP

swift imp
#

Has anyone worked with __class_getitem__?

#

Documentation and examples on it are really sparse

raven ridge
#

I know of it, but I've never needed it for anything. As far as I know it's really only used for generics

#

And the implementation is usually just that the class returns itself

paper echo
#

i don't think there's even a point in using __class_getitem__ unless you want to confuse the hell out of people. mypy won't know what to do with it anyway, right?

pliant tusk
#

i use it for some class constructors

deft pagoda
#

i used it as a class constructor for ranges

#

convenient to pass slices to a class

pliant tusk
#

(MyArray*5)[int] would return MyArray_5_int for example

deft pagoda
# 
In [141]: Range[2:]
Out[141]: Range(2, ∞, start_inc=True, end_inc=False)

In [142]: Range[2:] ^ Range[6:]
Out[142]: Range(2, 6, start_inc=True, end_inc=False)

This was just so i could use [] instead of () cause i don't like typing parens

swift imp
#

But I was thinking of using it to use annotations for some custom system

#

We have some dataframe subclasses and I want to use it to indicate we want to slice these columns out of the instance of this class before passing it to this argument

#

Total abuse but like idgiaf

#

Typing is a fool's errand anyway

#

Basically


def foo(a: A[[1,2]]):
    ...

Upon inspection we would have (A, [1,2]) and then we do A(<data passed to a argument>)[[1,2]] when we call foo

paper echo
#

it was recently brought to my attention that "annotations" were originally not meant only for types, but for generic metadata

#

so it's not abuse as such

spark magnet
swift imp
unkempt rock
# 
class Solution:
    def threeSum(self, nums: List[int]) -> List[List[int]]:
        res, dups = set(), set()
        seen = {}
        for i, val1 in enumerate(nums):
            if val1 not in dups:
                dups.add(val1)
                for j, val2 in enumerate(nums[i+1:]):
                    complement = -val1 - val2
                    if complement in seen and seen[complement] == i:
                        res.add(tuple(sorted((val1, val2, complement))))
                    seen[val2] = i
        return res

how is the answer returned as a List of list(s) here?
why not a set?

spark magnet
unkempt rock
#

the python prompt lol

spark magnet
unkempt rock
#

why

spark magnet
# unkempt rock why

it doesn't return a list of lists. it returns a set. Why do you think it returns a list of lists?

unkempt rock
#

it does when I run the code

#

also the code comes from an official source

spark magnet
#

can you show us the output you are seeing?

unkempt rock
spark magnet
#

That could be their test runner converting your result into a list of lists. Try adding print(res) just before your return statement

unkempt rock
#

print gives {(-1, 0, 1), (-1, -1, 2)}

#

that's really odd for them to convert it at run time tho

spark magnet
#

@unkempt rock it's also odd for them to insist that you put your code into a class called Solution. This smacks of Java.

unkempt rock
#

this is so they can create an object to run their test cases I believe

spark magnet
unkempt rock
#

also sometimes you need to instantiate or initialize a data structure that's why

spark magnet
spark magnet
unkempt rock
#

well idk

spark magnet
#

@unkempt rock right, leetcode always gives you a class to fill out. it's kind of silly, but harmless.

swift imp
swift imp
#

How do you get information from Annotated

#

__metadata__

grave jolt
#

Strangely enough, in Rust they provide a snake_case name, but that's because the compiler will give you a warning if it's in camelCase

unknown.png
#

And in TypeScript, they don't require you to make a class?..

unknown.png
#

anyway, sorry for the off-topic

rich cradle
grave jolt
#

I still don't understand why Python solutions require a class. "It's an OO language, so we have to stick a class somewhere in there"?

rich cradle
terse orchid
#

Hello. Does anyone interested in discussing a project and nosql db architecture I'm working on?

#

I mean, real discussion, if anyone's available

spark magnet
#

@terse orchid it works best to start talking, and see if people join in.

terse orchid
#

I just didn't wanted to occupy the channel 🙂 anyway;

#

The project is about to collect various data about companies from the internet (public data), and let the customer monitor social media posts and news about both themselves and their competitors

#

The thing is this project is already working. But it got too complicated. Because there are different sources and customers may have different demands. It (the db) has to be very flexible. One tough point is, two companies may conflict. I mean, both of them are using the product and both of them are competitors to each other

#

Currently the project is running with mongodb as database, flask as API server. That's pretty much it

#

I know it's hard to join the topic, it's okay. I just wanted to share because I'm a bit overwhelmed 🙂 It's all right if nobody answers

warm mango
#

I wrote a dependency injection library with which you can register certain types in a container and automate instantiation of them
Example:

# container.py
import main
from python_di import containers, providers

class Container(containers.DeclarativeContainer):
    a = providers.Factory(main.A)
    b = providers.Factory(main.B)

container = Container()
container.wire(modules=[main])

if __name__ == '__main__':
    main.function()

# main.py
from python_di.wiring import inject

class A:
    pass

class B:
    def __init__(self, a: A):
        self.a = a

@inject
def function(a: A, b: B):
    print(a, b)

If any dependencies have type hints in their signatures (__init__ in case of classes) they would be provided from container
Also dependencies could be registered as abstractions, so for example:
Factory(IService, Service) if some of classes would need IService Service would be provided instead.
Is there anything i can add into it?

tidal marten
#

How do you know which object/class to inject?

#

Using globals?

warm mango
#

@tidal marten By function signature type annotations

#

Also all providers should be registered in Container directly or via Module,
Providers on the other hand accept type and callable

class Factory(TypedProvider, Generic[T]):
    def __init__(
        self,
        type_: Type[T],
        factory: Callable[..., T] = None,
        **kwargs,
    ):

And factory defaults to type_

tidal marten
#

Ohh, so the signature actually have reference to the original object/class. That's neat...

warm mango
#

It would not work if you're importing via if TYPE_CHECKING block, but that would be impossible to inject too 🤔

#

Plus you could avoid that using abstract classes

warm mango
tidal marten
#

Well, I haven't had any need for such thing (in Python). But I have encountered it in Angular.

warm mango
#

Yeah, that's something similar

tidal marten
#

Probably if you have a system that needs abstraction like Angular, then sure

warm mango
#

Most systems could benefit from abstractions i think

tidal marten
#

I'm just wondering whether setting up this abstraction is worth the additional boilerplate stuffs

warm mango
#

There's really no boilerplate i think pithink

#

You don't have to create interfaces, you could register classes as-is if you want*

tidal marten
#

What about the initialization though?

warm mango
#

Hm?

tidal marten
#

My understanding (for Angular at least) is that the dependency injection is used to inject dynamic state object into the class

#

So I'm wondering if this is the case with your library

warm mango
#

What do you mean by "dynamic state object"?

#

Any class/object can be injected

tidal marten
#

Like for example, if I want to do this with Flask, I'd do something like:

@inject
def route_index(route: Route):
    ...

and the route would be filled with the current route data and utilities

warm mango
#

What is that Route class?

#

afaik most of flask objects are thread-bound, right?

tidal marten
#

Well, in actuality, Flask use decorator to register a route and it will just pass in some argument to your corresponding function, so it doesn't actually use dependency injection I think.

gleaming rover
#

but the idea of dependency injection is really just that whatever needs stuff gets it passed to it rather than making it itself

warm mango
#

Anyway, flask would pass route parameter to your function if you declare it in it, right?

warm mango
tidal marten
#

Hmm, let me see. Maybe this dependency injection can be used for something like:

@inject
def handle_request(request: Request, db: DatabaseConnection):
    ...

So the dependency injection would automatically supply the Request and DatabaseConnection objects automatically like that?

warm mango
tidal marten
#

But your factories for those injected data would need to depend on some sort of a global/singleton then

#

I mean, that could work

warm mango
#

container? Yeah, there should be only one 🤔

#

But it doesn't have to be this way

tidal marten
#

Like for DatabaseConnection, you will need to have a pool of connection stored somewhere

warm mango
tidal marten
#

So in your __main__ you initialize DatabaseConnectionFactory with some database connection, then register the factory to the container?

warm mango
# 
class DbConnection:
    pass

class Container(containers.DeclarativeContainer):
    db_connection = providers.Singleton(DbConnection)

More like this pithink

tidal marten
#

Hmmm... 🤔

#

So you can just call handle_request() without passing any arguments?

warm mango
#

I think so

#

If you register that flask-specific Request class somehow - yeah

tidal marten
#

Not sure if I like the implicit nature of this, lol. But this can be useful if you have a massive project.

warm mango
#

Since both container and request handler would run in same thread you could probably just get flask.globals.request

warm mango
tidal marten
#

Yeah, but using globals are kinda meh (in Flask-way)

warm mango
# 
@inject
def handle_request(
    request: Request = Provide, 
    db: DatabaseConnection = Provide,
):
    ...
warm mango
# 
class Container(containers.DeclarativeContainer):
    db_connection = providers.Singleton(DbConnection)
    flask_request = providers.Factory(Request, lambda: flask.request)
    # or since it's a proxy maybe we could do it this way
    flask_request = providers.Object(Request, flask.request)
tidal marten
#

So, in this situation, the function handling the routing does not need to pass the arguments. But, the function, through the decorator, can request what things it needs.

warm mango
#

For each argument if such type is registered in a container it would inject it

tidal marten
#

And, for instance, Flask may only provide Request and Response injection. I assume you can add third-party module to provide e.g. SQLAlchemyConnection or RedisConnection as well?

#

But in this instance you need to control the initial declaration of the injection (the Container), I assume

warm mango
#

Also since you need some sort of arguments for them e.g. host, port you could provide them as well

tidal marten
#

Quite neat

warm mango
# 
from python_di import containers, providers

class RedisClient:
    def __init__(self, host: str, port: str):
        pass

class Container(containers.DeclarativeContainer):
    config = providers.Configuration()
    redis_client = providers.Singleton(
        RedisClient,
        host=config.redis.host,
        port=config.redis.port,
    )

container = Container()
container.config.from_dict({
    "redis": {
        "host": "localhost",
        "port": "I Don't know what port redis uses by default :)"
    }
})
tidal marten
#

I assume the injector instantiates new object for each function call as well?

warm mango
#

But they're reused during single injection

# 
@inject
def function_to_wire(a: A, b: B) -> Tuple[A, B]:
    return a, b

def test_instances_are_reused_during_single_injection(module_to_wire):
    container = _Container()
    container.wire(modules=[module_to_wire])
    a, b = module_to_wire.function_to_wire()
    assert b.a is a
# 
class A:
    pass

class B:
    def __init__(self, a: A):
        self.a = a
#

That behavior is easy to change by implementing different resolver class

#

Container is merely a storage for providers, all work for resolving types is delegated to Resolver pithink

unkempt rock
#

I made this design using turtle:

image0.jpg
#

Thanks! It uses the turtle circle function and I programmed it to where every-time a circle is made, it will move the turtle up one pixel and it increases the size of the circle by 1.

#

Nope, I just hold in one hand and the computer in the otber.

#

Here’s the code btw:

import turtle as t

def a():
    t.hideturtle()
    t.penup()
    t.goto(0, -255)
    for i in range(0, 1):
        for o in range(0, 1000):
            t.pendown()
            t.speed(1050)
            t.circle(o)
            t.penup()
            t.speed(1050)
            t.circle(o)
            ax = “black”
            t.color(ax)
    t.penup()

while True:
    a()
sage siren
#

Hey idk if this is the right channel to ask this, but is there some lib which I can use to convert a gif to apng?
Like if no lib then any piece of code example would be lovely too

snow badge
#

I think ffmpeg could also be of help

#

@sage siren

sage siren
#

Wow thanks, I will take a look : )

frigid hinge
#

Hello guys, I do not know if this goes here or not, but I am looking for some library that focuses on optics (I am working with mirrors) and another in solar resources (Ray Tracing, Sunshape, ...). I hope you can help me in my search: D

unkempt rock
# 
if A[i] != x and B[i] != x:
#

how would you use De Morgan's laws to translate this

#

into using "or" (making it more compact)

#

or is it not possible to make it much more compact?

charred pilot
#

not a and not b == not (a or b)

unkempt rock
#

not ((A[i] != x) or (B[i] != x)) ?

warm mango
#

That's what matters

radiant fulcrum
#

or easier: x not in {A[i], B[i]}

raven ridge
#

that assumes hashability - use a tuple, not a set.

radiant fulcrum
#

true

charred pilot
#

probably faster than making a set anyway

raven ridge
#

that too.

unkempt rock
#

if x is not in either A[i] or B[i]

raven ridge
#

de morgan's law would be if A[i] == x or B[i] == x, but if x in (A[i], B[i]) would be the more succinct, readable way to write that.

charred pilot
#

huh, what?

radiant fulcrum
#

also means if you change the variable name, you're not altering a massive line of variables or if you want to add another thing to have x not in

raven ridge
#

er, I missed a not in both of those.

charred pilot
#

ah, makes more sense then

raven ridge
#

de morgan's law would be if not (A[i] == x or B[i] == x), but if x not in (A[i], B[i]) would be the more succinct, readable way to write that.

#

should be that, I think 🙂

charred pilot
#

the first was already good, tbh

raven ridge
#

yep. OK, third time's the charm.

charred pilot
#

isn't it not (A[i] == x or B[i] == x)?

raven ridge
#

stupid de morgan

charred pilot
#

oh well. optimize for readability, not code size

raven ridge
#

of the 3 choices, either if A[i] != x and B[i] != x: or if x not in (A[i], B[i]) seem most readable.

#

if not (A[i] == x or B[i] == x) is definitely harder to wrap your head around.

unkempt rock
#

the != x was confusing me

unkempt rock
#

if you translate it into English

#

"if A[i] or B[i] doesn't equal x"

#

but yea the doesn't part is on the outside

rich cradle
#

This server doesn't allow job postings or recruiting of any kind.

#

!rule 9

fallen slateBOT
#
Rules

9. Do not offer or ask for paid work of any kind.

ornate ginkgo
#

Sorry, I’ll delete it

#

Technically it isn’t paid work but I’ll respect it

paper echo
#

It depends on what the "condition" is i think

tidal marten
#

How do you know if Python really uses like 1GB of RAM or it's just a bad case of memory fragmentation? Lol

grave jolt
tidal marten
#

I'm parsing a 60'000-line file into some sort of tree, lol

#

And it took like 12 second for the whole tree to be constructed

eager trail
#

I had a function I was supposed to rework take 3.5gb of ram to run because it was trying to hold 30k nested dicts from an api call in memory, after I changed it to paginate and yield only the important bits it dropped to 250mb

tidal marten
#

Most of my nodes only hold the start and end index of the initial buffer though, so I expect that there should be no duplication issues. Maybe some nodes hold onto regex match objects and that have duplicate substrings?

#

Is there any profiler that I can use to try pin things down?

unkempt rock
undone oyster
#

Hello. Has anyone ever worked with Python on webrtc?

quartz garnet
#

can anyone pls tell that pandas datareader for yahoo.finance is working or not?

#

Traceback (most recent call last):

pandas_datareader._utils.RemoteDataError: Unable to read URL: https://finance.yahoo.com/quote/TSLA/history?period1=1356993000&period2=1577917799&interval=1d&frequency=1d&filter=history
Response Text:
b'<!DOCTYPE html>\n <html lang="en-us"><head>\n <meta http-equiv="content-type" content="text/html; charset=UTF-8">\n <meta charset="utf-8">\n <title>Yahoo</title>\n <meta name="viewport" content="width=device-width,initial-scale=1,minimal-ui">\n <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">\n <style>\n html {\n height: 100%;\n }\n body {\n background: #fafafc url(https://s.yimg.com/nn/img/sad-panda-201402200631.png) 50% 50%;\n background-size: cover;\n height: 100%;\n text-align: center;\n font: 300 18px "helvetica neue", helvetica, verdana, tahoma, arial, sans-serif;\n }\n table {\n height: 100%;\n width: 100%;\n table-layout: fixed;\n border-collapse: collapse;\n border-spacing: 0;\n border: none;\n }\n h1 {\n font-size: 42px;\n font-weight: 400;\n color: #400090;\n }\n p {\n color: #1A1A1A;\n }\n #message-1 {\n font-weight: bold;\n margin: 0;\n }\n #message-2 {\n display: inline-block;\n *display: inline;\n zoom: 1;\n max-width: 17em;\n _width: 17em;\n }\n </style>\n <script>\n document.write('<img src="//geo.yahoo.com/b?s=1197757129&t='+new Date().getTime()+'&src=aws&err_url='+encodeURIComponent(document.URL)+'&err=%<pssc>&test='+encodeURIComponent('%<{Bucket}cqh[:200]>')+'" width="0px" height="0px"/>');var beacon = new Image();beacon.src="//bcn.fp.yahoo.com/p?s=1197757129&t="+new Date().getTime()+"&src=aws&err_url="+encodeURIComponent(document.URL)+"&err=%<pssc>&test="+encodeURIComponent('%<{Bucket}cqh[:200]>');\n
</script>\n </head>\n <body>\n <!-- status code : 404 -->\n <!-- Not Found on Server -->\n <table>\n <tbody><tr>\n <td>\n <img src="https://s.yimg.com/rz/p/yahoo_frontpage_en-US_s_f_p_205x58_frontpage.png" alt="Yahoo
Logo">\n <h1 style="margin-top:20px;">Will be right back...</h1>\n <p id="message-1">Thank you for your patience.</p>\n <p id="message-2">Our engineers are working quickly to resolve the issue.</p>\n </td>\n </tr>\n </tbody></table>\n </body></html>'

Tesla, Inc. (TSLA) Stock Historical Prices & Data - Yahoo Finance

Discover historical prices for TSLA stock on Yahoo Finance. View daily, weekly or monthly format back to when Tesla, Inc. stock was issued.

#

Pls help me getting this error

hasty swift
#

<p id="message-1">Thank you for your patience.</p>\n <p id="message-2">Our engineers are working quickly to resolve the issue.</p>\n

#

sure sounds like yahoo's servers are telling you they have an issue and are asking you to be patient

light estuary
#

Can you implement dynamic method variations based on the content of a class? Let's say i have a datatype that can hold either a list or dict. and i want different methods available based on that.

def add(self, item):
    self.data.append(item)
###
def add(self, key, value):
    self.data[key] = value
hasty swift
#

you'll generally want to avoid such things as it can be confusing, also in Python you can only have one function definition with one name so the second def add would override the name add

#

you CAN do logic in a function to check the types of arguments, but it's generally considered unpythonic

spice pecan
#

You technically can, but you really shouldn't

hasty swift
#

there are some rare cases where you might want to implement logic like that using e.g. instanceof but yeah, avoid it

light estuary
#

Even if i agree, i want to try it out here. So how would you make conditional method declarations at init time? Can i just wrap them in an if statement?

spice pecan
#

You can check a cached bool that tells you whether you're holding a dict, run an isinstance check, or do something else to determine what to do with the arguments. You could also create a baseclass with shared logic and then two subclasses, one of which holds a dict and the other a list

spice pecan
hasty swift
#

anyway there's a whole metaprogramming thing with Python that may help you, it gets pretty confusing pretty quickly tho

spice pecan
#

If you really have to, I'd suggest overriding the method in a subclass, as that would make the code significantly more readable

light estuary
#

I think it becomes messy if you handle multiple unique cases in the same function. when you can separate them into their own contexts

hasty swift
#

yes, so why are you trying to give them the exact same name, but different parameter count, and argument types?

#

THAT is also very unclear

spice pecan
#

Implementing function overloads has the unfortunate side-effect of complicating traceback, which would be avoided with method overrides. Other than that, you could just do

def add(self, arg1, arg2=None):
    if isinstance(self.data, dict):
        self.data[arg1] = arg2
        return
    self.data.append(arg1)
light estuary
#

Sure, but none of this will be exposed to the user. The idea is that i want to offer one type that can handle multiple different data types.

hasty swift
#

you might want a factory

spice pecan
#

Since python is dynamically typed, what benefit would there be to using a single type over a factory that gives you different types?

hasty swift
#

anyway, the most common way I think people would solve something like this is

class ListManager():
  def add(self, some, things):
    pass

class DictManager():
  def add(self, some):
    pass

def get_manager(arg):
  if isinstance(arg, list):
    return ListManager()
  if isinstance(arg, dict):
    return DictManager()
  raise NotImplementedError()

foo = [] or {}
manager = get_manager(foo)
manager.add(foo)

or something along those lines

spice pecan
#

If you really wanted to, you could override __new__ in a base class, but like... ew

hasty swift
#

I've done some static methods that iterate through __subclasses__ and return based on some match

light estuary
#

It might not be the best idea. But it's worth trying new ideas. I'm used to only using statically typed languages. so the dynamic nature of Python makes for some really interesting possibilities.

hasty swift
#

well there's dynamic and there's confusing

spice pecan
#

Having those possibilities doesn't necessarily mean you should abuse them

light estuary
#

I don't disagree. but you can't know what is a bad idea until you've tried it.

spice pecan
#

For example, what would happen if you passed two arguments to a list manager? Would you silently swallow the second argument, or throw an error?

hasty swift
#

really there's no problem with just adding the type to the function name

add_dict(...)
add_list(...)
#

clear, well documented, doesn't confuse IDEs, etc.

unkempt rock
spice pecan
spice pecan
#

Though I guess it's not that bad

hasty swift
#

imo things like that are good left to the rare libraries that really provide sensible functionality with it

spice pecan
#

Some of the design decisions in pathlib are... interesting

#

Path concatenation via __truediv__, for example

hasty swift
spice pecan
#

Strangely, I'm not opposed to that one

hasty swift
#

I can .. kind of see why someone would think that reduces boilerplate, but it definitely is surprising

spice pecan
#

But it's uncommon to say the least

hasty swift
#

the principle of least astonishment is definitely violated with that

#

but it's .. probably still a good choice

spice pecan
#

It looks faithful to the path representation at least

hasty swift
#

on *nix LUL

spice pecan
#

And I might be wrong on this one, but IIRC + does the same thing, so at least they don't remove the obvious way to do it?

#

!e ```py
from pathlib import Path

print(Path('/') + 'home')

fallen slateBOT
#

@spice pecan :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 3, in <module>
003 | TypeError: unsupported operand type(s) for +: 'PosixPath' and 'str'
spice pecan
#

I am indeed wrong on that one

hasty swift
#

now that pathlib is in stdlib I've gotten pretty used to use it for all kinds of small things that make things easier, like .. reading or writing simple text contents

light estuary
#

Since you were curious. I'm making a small helper library for a menu system. On the users end it will look like this.
Any default data type with items that are callable will be allowed.

from menu import Menu

mpd = Menu({
   "play/pause": mpd.toggle(),
   "play"      : mpd.play(),
   "stop"      : mpd.stop(),
   "next"      : mpd.next(),
   "prev"      : mpd.prev()
})

start = Menu([
    run("firefox"),
    run("pcmanfm"),
])

talk = Menu({
    echo("foo"),
    echo("bar")
})
hasty swift
#

Path(MY_CONFIG).read_text() is just quite a bit more convenient than with open() ...

spice pecan
#

Pathlib on its own is very useful, it's just that certain decisions are not ones you'd expect

spice pecan
hasty swift
light estuary
#

it uses the display implementation for the type @hasty swift

hasty swift
#

right

#

looks like you'll want to check isinstance in __init__

#

or something like that

#

and normalize

#

I'd then write similar isinstance checks in add if the idea is to add menu items with that, or do something based on argument count

#

so e.g. range() works so range(3) == range(0, 3) == range(0, 3, 1)

spice pecan
# 
class BaseMenu:
    def __init__(self, data):
        self.data = data


class ListMenu(BaseMenu):
    def add(self, item):
        self.data.append(item)


class DictMenu(BaseMenu):
    def add(self, key, value):
        if key in self.data:
            raise KeyError('The specified key is already present')
        self.data[key] = value


def Menu(data):
    if isinstnace(data, list):
        return ListManager(data)
    return DictManager(data)
light estuary
#

This makes me curious. in order to decrease confusion. Would it possible to override the type association and link the underlying data types methods through the class?
So you could use it as if it weren't wrapped? For example.

test = Menu({
   "123": Echo("123"),
})

test["456"] = Echo("456")
hasty swift
#

you can do various overrides on the magic methods like __setitem__ and __getitem__ or whatever they're called, but it's not exactly going to reduce confusion in this case

light estuary
#

This is all pretty interesting. thanks for the input

spice pecan
#

Since __getitem__ and __setitem__ have the same interface on all classes, you could implement it once and it would work for both dicts and lists

terse orchid
#

Would you follow camelCase or snake_case notation for API parameters that being used by a language that follows camelCase? E.g. a flask api that being used by Angular

unkempt rock
#

@terse orchid I have thought about the same thing tbh

#

but afaik REST is supposed to be camelCase

#

irrelevant of the backend language

terse orchid
#

Hmm I see, you are right

unkempt rock
#

in general, I as a user prefer to see APIs follow camelCase but thats actually a good question haha

terse orchid
#

I will continue with the camelCase 🙂

unkempt rock
#

uh any idea where the free help channels are?

#

im doing some tensorflow and ran into a massive issue that is "fixed" tahts what the git says 🤣

terse orchid
paper echo
vagrant basin
#

why do i have this when i run python in cmd

Screenshot_13.png
#

i did whatever i could

lean basin
#

hello

finite sparrow
#

@frigid mortar so are you making your own language, or interfacing with other languages?

frigid mortar
#

Recreating Python (and various other languages) API within the D programming language.

raven ridge
#

It sounds like you intend to recreate the Python standard library, as well. Reimplementing enough of the language that the standard library will work is a huge amount of work.

finite sparrow
#

theres certain things python does that i would like to see in other languages, but a lot of the things are going to be so hard to do that you would have to basically create another language within the language.

frigid mortar
#

something along the lines of doing this:

module App;

import Python;

void main(string[] args)
{
    print("Hello World!");
}

Where generally by importing Python u can start accessing functions (by name/space and casing) as youd expect to find them from the base lang.

raven ridge
#

are you planning on linking against an existing Python interpreter, or creating your own?

finite sparrow
#

hm, this would only allow access to python builtin functions? the smallest part of python that i like is the functions, a lot of it is the syntax for which you need a interpreter.

frigid mortar
#

The most direct way to describe it is:
Its about reducing the amount of actual learning of another language to be productive.

finite sparrow
#

how would you recreate things like comprehensions (assuming they are not in the language already)?
what about dunder methods? are there equivalents to these concepts that you can translate to in D?

frigid mortar
#

got an example of what those are?

finite sparrow
#

!listcomp

fallen slateBOT
#

Do you ever find yourself writing something like this?

>>> squares = []
>>> for n in range(5):
...    squares.append(n ** 2)
[0, 1, 4, 9, 16]

Using list comprehensions can make this both shorter and more readable. As a list comprehension, the same code would look like this:

>>> [n ** 2 for n in range(5)]
[0, 1, 4, 9, 16]

List comprehensions also get an if statement:

>>> [n ** 2 for n in range(5) if n % 2 == 0]
[0, 4, 16]

For more info, see this pythonforbeginners.com post.

frigid mortar
#

im unfamiliar with that concept specifcially o.o

finite sparrow
#

!e ```py

class A:
def init(self, num):
self.num = num
def add(self, other):
return A(self.num + other.num)
def repr(self):
return f"<A {self.num}>"

one = A(1)
two = A(2)
print(one + two)

fallen slateBOT
#

@finite sparrow :white_check_mark: Your eval job has completed with return code 0.

<A 3>
finite sparrow
#

as you can see this modifies the way addition works by defining a special method in a class.

#

this is the way you do operator overloading in python. you can also observer __repr__ which is the string representation of a class

frigid mortar
#

dlang has operator overloading 😛

raven ridge
#

but there's the reversed addition operator __radd__, which isn't a concept that exists in most languages.

frigid mortar
#

and accourding to some page i was looking at we got some weird crap with array crafting o.o

#

had no idea that existed

finite sparrow
#

the core issue remains: how will you replicate syntax specific to python in a entirely different language?

frigid mortar
#

i cant exactly replicate 'syntax'... but its not exactly my goal to either.

finite sparrow
#

hell, D is statically typed right? how will you deal with how python handles types?

finite sparrow
raven ridge
#

!e ```py
class A:
def init(self, val):
self.a_val = val
def add(self, other):
return NotImplemented

class B:
def init(self, val):
self.b_val = val
def radd(self, other):
return B(self.b_val + other.a_val)

print((A(5) + B(10)).b_val)

fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

15
frigid mortar
#

the fact ur bot will print results is amazing o.o

raven ridge
#

note here that we've got an A on the LHS and a B on the RHS of a +. The A says "I don't know how to add", then the B gets a chance to define how to add.

finite sparrow
raven ridge
#

B.__radd__ is called if and only if A.__add__ returns NotImplemented. That's different from how operator overloading works in a language like D, where I believe the appropriate equality comparison method is resolved statically at compile time.

frigid mortar
finite sparrow
raven ridge
#

You can also already write a Python module in C or C++ or Rust.

#

Probably D, too.

frigid mortar
#

i had no idea that existed o.o

raven ridge
#

If D can produce shared libraries with a C ABI, then you can write Python modules in D.

frigid mortar
#

it can

#

extern(C):

raven ridge
#

thought so.

finite sparrow
#

the main thing that would make me switch (more like, learn another) language is not that the other language replicates python, i already have python for that after all. its that the other language can offer me something else that i value enough to learn the language.

frigid mortar
#

ic

finite sparrow
#

for example i learned a bit of HLSL because it provides me with something VASTLY different from python, mainly highly parallelised computation on the GPU

#

something else that could make me learn another language is easy native (not electron) cross platform GUIs

frigid mortar
#

What if there were tools in the language you may want to interface with but dont have good compatibliity with python directly?

raven ridge
frigid mortar
finite sparrow
# frigid mortar What if there were tools in the language you may want to interface with but dont...

another language doesn't need to be compatible with python. HLSL is basically impossible to get to work with python, hence i learned a bit of C# to be able to even run the HLSL. and by learned i mean that i googled syntax for a bit because at a certain point the only thing you need to look up is syntax, which is not a hard thing to do.

if i were to learn another langauge, i would want something vastly different from python. if i can write python extensions in it, neat, but thats not a requirement by far. the language would be one that can be used on its own.

raven ridge
#

if i were to learn another langauge, i would want something vastly different from python.
+1 on this. In my daily life I regularly switch between C++, Python, and shell - languages are more useful to me if they satisfy a radically different niche than one of the ones I'm already comfortable in.

#

If it's doing something I can already do in one of those languages, I'd just use that language.

frigid mortar
#

maybe im just used to crappy devs then xD I started the multiple projects i have because the communities i grew up refused to even try new stuff if it meant starting from 0... having the functions exist was a compromise they said at least made it viable cause they could immediately use syntax they were learning in a useful way rather than experiment for weeks before knowing how to do anything specificly.

finite sparrow
#

i hope that helps you understand why i think the whole idea of porting another languages functions over is not really great. beacause it often doesnt matter. why do i care that my way of printing out is println!("hey") rather than print("hey")? why does it matter that i have to import java.lang.Math; instead of import math? in the end those are small syntax differences, which dont matter when compared to the underlying structures. i dont like java because of its approach to OOP, which is not something someone can fix by making a library of functions that have the same name as in python. i have no opinion on D, this is just a example to show what i really consider important.

raven ridge
#

and moreover: if you advertise "Python compatible" as a feature, you'll be forever fixing bugs that you don't care about at all because people find things that aren't actually Python-compatible.

finite sparrow
frigid mortar
raven ridge
#

Python is a weird language. Names aren't resolved when the file is compiled, or even the point where the module is imported, but at the point where the function is called.

#

You'll find implementing your own Python-compatible language isn't trivial. It works very differently than most languages.

frigid mortar
#

I wouldnt have concidered trying if Dlang werent also fekin weird

finite sparrow
#

you seem to like D, Awsome!
make projects in D, and if you want to work with people that are unwilling to use D why not let your part of a project and their part of a project communicate over a language agnostic protocol (API, message queue, etc) so that they can use whatever they want.

frigid mortar
#

it has some features inspired heavily by scripting langs

raven ridge
#

!e ```py
def to_string(i):
return str(i)

print(to_string(42))
str = print
to_string(100)

fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

001 | 42
002 | 100
finite sparrow
raven ridge
#

maybe, but things like that are used all the time by unit tests.

#

!e ```py
def n_adder(n):
def add_n(x):
return x + n
return add_n
add5 = n_adder(5)
print(add5(add5(2)))

fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

12
tawdry ruin
raven ridge
#

I mean, don't do that. But mocks do that.

frigid mortar
#

that is a very weird function o.o

finite sparrow
frigid mortar
#

for a func inside a func?

raven ridge
#

that's a closure. I'm sure D has them, but it's an example of a way to capture variables from outer scopes in Python.

frigid mortar
#

ah

raven ridge
#

Which also serves to demonstrate some of Python's name lookup weirdness.

frigid mortar
#

that is spooky either way xD

raven ridge
#

the way Python looks up names is very different than how most other languages do it. Most languages resolve a name to a memory address at compile time, Python does it each time the function is called.

finite sparrow
#

makes me wonder, is there a way to force python to bypass this, to improve performance?

raven ridge
#

Cython has an option for that.

finite sparrow
#

yeh well, thats c++ in the end not python

raven ridge
#

right. Python doesn't have a way to do that.

finite sparrow
#

hmm, i'll have to take a look at the bytecode, maybe i can make something truly awful out of this idea

raven ridge
#

if you want to get ugly, there is a tricky way to early bind: default arguments for a function are bound early.

finite sparrow
#

oh

#

oh fun

raven ridge
#

it used to be advised as a Python performance hack

#

it's still used in some places in the stdlib.

native flame
finite sparrow
raven ridge
fallen slateBOT
#

Lib/sre_compile.py lines 411 to 414

def _mk_bitmap(bits, _CODEBITS=_CODEBITS, _int=int):
    s = bits.translate(_BITS_TRANS)[::-1]
    return [_int(s[i - _CODEBITS: i], 2)
            for i in range(len(s), 0, -_CODEBITS)]```
raven ridge
#

forcing early binding of int

#

as well as _CODEBITS

sand python
#

why does that improve performance?

finite sparrow
#

OHHH ive used that before, in my thing where i tried to replicate classes purley with decorators.

raven ridge
# sand python why does that improve performance?

python looks names up in the locals scope, then the enclosing scope(s), then the global scope, then the builtins scope. This makes it so that a lookup of _int against the locals can find int, saving the need for a failing lookup against locals followed by a failing lookup against the enclosing scope followed by a successful lookup against the globals

sand python
#

mm, got it

raven ridge
#

but the general consensus these days is that you shouldn't micro-optimize Python code like that - if it's performance sensitive enough to justify that microoptimization, it probably should be written in C instead.

radiant garden
#

Haha, sometimes you forget about the overhead of iterating on the python end, and sometimes you get a 30fold performance increase by passing a numpy array into a c function instead

grave jolt
finite sparrow
grave jolt
#

no, I'll look at it later

#

have to go now

craggy birch
#

I need some help with n-gram model in ML

#

Given the headline of the news, the objective is to identify the category of news.
Note: Use the only headline as input and predict the
News_Category_Dataset.json
category

For example:
Short_description: ....
Headline: There Were 2 Mass Shootings In Texas Last Week, But Only 1 On TV
Date : ....
Link : ....
Authors: ....
Category: CRIME
Consider the following categories only: Business, Comedy, Sports, Crime, and Religion

The Features of this dataset is described as:
a. Lexical Features: Word n-gram.
b. Syntactic Features: Parts of speech tag unigrams.
Implement n-gram (n=1,2 and 3), called unigram, bigram and trigram features for each instance. You may choose only the most frequent n-gram to provide as a features for your model. For n=1, use 500 most frequent unigram, similarly use 300 and 200 most frequent bigram and trigram respectively.
For parts of speech tag unigrams, first you need to get POS tag for each instance. You can use any library like Stanford POS tagger. Find the following link for it:
https://nlp.stanford.edu/software/tagger.shtml

For NLTK POS tagger see
http://www.nltk.org/book/ch05.html

Similar to lexical feature use 500 most frequent unigram to build the model.

Result and Evaluation: Perform 10-fold cross-validation and report the performance of the classification models: Decision Tree, SVM, ANN and Maximum Entropy. Analysis the result with different performance evaluation metrics.
Dataset: news-category-dataset.json

The Stanford NLP Group

Performing groundbreaking Natural Language Processing research since 1999.

fallen slateBOT
#

Hey @craggy birch!

Uh-oh! It looks like your message got zapped by our spam filter. We currently don't allow .json attachments, so here are some tips to help you travel safely:

• If you attempted to send a message longer than 2000 characters, try shortening your message to fit within the character limit or use a pasting service (see below)

• If you tried to show someone your code, you can use codeblocks
(run !code-blocks in #bot-commands for more information) or use a pasting service like:

https://paste.pythondiscord.com

sand python
craggy birch
#

okay thanks

tidal marten
#

You guys should hide this channel somewhere people won't be able to look up easily, lol

woven robin
#

When writing a __eq__ method for a class, does it make sense to use a -> bool type hint for the return? If not, why not?

flat gazelle
#

strictly speaking, the return type is bool | NotImplemented

#

but if you never return that, it should be fine

woven robin
#

Pre-3.9, would that be typing.Union[bool, NotImplemented]?

flat gazelle
#

yes

#

oh wait, NotImplemented is not considered a type hint

woven robin
#

Where did you read that?

flat gazelle
#

it seems to something of a mess

woven robin
flat gazelle
#

yes

#

you can return NotImplemented from almost any dunder to notify python that you don't know how to handle the type you were given

woven robin
#

ohh

flat gazelle
#

and python will try to ask the other type, and if that fails, it will error out

#

it is pretty important to do so as often as possible, since it makes it easier to integrate your types into other codebases

woven robin
#

Why do you return NotImplemented instead of raising NotImplementedError?

flat gazelle
#

they behave slightly differently

woven robin
#

What behavior does returning NotImplemented offer that is desirable over raising NotImplementedError?

flat gazelle
#

!e

class A:
    def __eq__(self, other):
        if isinstance(other, A):
            return True
        raise NotImplementedError()
class B:
    def __eq__(self, other):
        if isinstance(other, B):
            return True
        return NotImplemented
class C:
    def __eq__(self, other): return True

print(f'{B() == C() = }')
print(f'{A() == C() = }')
fallen slateBOT
#

@flat gazelle :x: Your eval job has completed with return code 1.

001 | B() == C() = True
002 | Traceback (most recent call last):
003 |   File "<string>", line 15, in <module>
004 |   File "<string>", line 5, in __eq__
005 | NotImplementedError
flat gazelle
#

NotImplementedError will not ask other types

safe hedge
#

NotImplementedError is mainly used on method of abstract base classes?

woven robin
# flat gazelle !e ```py class A: def __eq__(self, other): if isinstance(other, A): ...

In the first print statement, why do both B's and C's __eq__ methods get called. I expected that only one of their methods would get called.

>>> class B:
    def __eq__(self, other):
        print("B's __eq__ called")
        if isinstance(other, B):
            return True
        return NotImplemented

    
>>> class C:
    def __eq__(self, other):
        print("C's __eq__ called")
        return True

    
>>> print(f'{B() == C() = }')
B's __eq__ called
C's __eq__ called
B() == C() = True
flat gazelle
#

it asks B() if it is equal to C(), B() answers I don't know = NotImplemented, so python asks C() instead, which answers yes=True

woven robin
#

ohhh

#

Thank you.

next oriole
#

how to select foreign key by session in django

urban charm
#

Hey all

#

im working on a project where i need a relationship between my two objects

#

sth like the django foreignkey relationship

#

where one object can access the stuff of another by just like --> object1.name_of_object2

#

or if thats not possible, how could i communicate between my module

#

like django signals where you can execute functions using the signals like post login and etc

#

how can i go about creating either one of these

gleaming rover
#

because bool has no subtypes and __eq__ is inherited

#

well not that it wouldn’t make sense

#

more…redundant?

native flame
#

__eq__s don't always return bools though

#

numpy arrays for example

# 
In [134]: np.array([1,2,3]) == np.array([1,2,4])
Out[134]: array([ True,  True, False])
raven ridge
#

At least mypy has a special case for NotImplemented - it's OK with you returning it from __eq__ despite declaring the return type as bool

#

Looks like it allows returning NotImplemented everywhere, and there's an open issue for restricting it to only be accepted in the contexts where the interpreter expects it. https://github.com/python/mypy/issues/363

unkempt rock
# 
dp = [0]*n
#

is there a point in using a dictionary ?

dp = {0: x for x in range(n)}
#

is it faster?

#

for dynamic programming

sand python
#

i think your comprehension is inverted? they all have key 0

#

I'd imagine a list would be faster if you just need an incrementing index

raven ridge
#

Inserting into a dictionary and appending to a list are both amortized O(1), meaning the performance difference between the two will be at most a constant factor. I expect the list to be faster overall, too.

unkempt rock
#

is the dp solution to fibonacci or similar problems equivalent in space and time complexity to the memoization solution?

raven ridge
gleaming rover
sand python
#

you rarely need to call __eq__ directly though, and you aren't going to get NotImplemented back from a == b even if both of their __eq__ return that.

wrt numpy arrays though, I guess it does violate it a bit, since to check for actual equality you'd have to do np.all(a == b), although numpy at least properly raises an exception if you try to bool(arr) and tells you how to get the behavior you want. a matter of practicality and readability for users I guess

unkempt rock
#

Тут есть русскиаязычные?

#

Here there are Russians?

sand python
#

yes, but we require that you use english in this server, sorry

unkempt rock
#

Ok, thank you for this information. I am going to using English. python

sage siren
#

You all know some lib which can be used for gif compression, have tried using PIL but the rendering is really bad 😔 😔

#

Have even tried wand, the rendering is good in this case but it's like super super slow and uses too much resources

light estuary
#

Can a type gain context over it's containment by traversing backwards?

class X(object):
    def y():    
        # calling A.b() <super.method()>

class A(object):
    def __init__(self, menu):
        self.x = X()
    def b():
        pass
#

To simplify the question. Can X figure out that's being stored inside of A?

swift imp
#

Is there documentation on object anywhere? I just recently learned about __subclass__ which captures all direct subclasses of a class that inherit from object and I had no idea of its existence.

peak spoke
light estuary
#

That's true. That will have to do.

spice pecan
#

You could use the descriptor protocol I guess

#

But that would only work when accessing it via the holder

swift imp
#

Well

#

So with the descriptor protocol you can capture type in __set_name__ but you don't know the instance until a __get__, __set__, or __delete__ access occur from the instance.

#

Maybe you could figure it out someway by capturing instances with __new__

#

With the way you are doing right now however you can never know

#

unless you like pass self to that X initialization

arctic crystal
#

Guys, what is the most hot niche of data science project or topics in freelance

swift imp
quasi hound
#

guys

#

this is going here cause i have a massive problem

#

/bug

#

check out this bit of code

# 
l = {msg: lambda: print(msg.upper()) for msg in ['a', 'b', 'c']}

while True:
  i = input()
  f = l[i]
  f()
#

you would think that it would print out any letter you give it as input, in uppercase form

#

but no

#

it always prints C

#

this is a simplified version of a problem i have in one of my projects

#

doing it without a list (or dict) comprehension works of course

#

if i manually did a: lambda(print('A')) and so on for each entry in the dict

#

but can anyone tell me why this doesn't

#

and so you guys don't have to waste time figuring out the code, the dict it gives looks like this

{
  'a': <lambda that should print 'A' but prints 'C'>,
  'b': <lambda that should print 'B' but prints 'C'>,
  'c': <lambda that prints 'C'>
}
``` and it's indexed with the user input over and over
#

weirdest shit i've ever seen

#

unless i'm missing something huge

astral gazelle
#

just closure behaviour i guess

spice pecan
#

Values outside of function's scopes don't persist, try using functools.partial instead

quasi hound
#

in what way

#

i've never used that

#

and thanks

#

but why does it always default to C

spice pecan
#

l = {msg: partial(print, msg.upper) for msg in ['a', 'b', 'c']}

astral gazelle
#

the lambda captures the msg variable from the outer scope but msg's value changes until it reaches c

spice pecan
#

!d functools.partial

fallen slateBOT
#
functools.partial 

functools.partial(func, /, *args, **keywords)```
Return a new [partial object](https://docs.python.org/3/library/functools.html#partial-objects) which when called will behave like *func* called with the positional arguments *args* and keyword arguments *keywords*. If more arguments are supplied to the call, they are appended to *args*. If additional keyword arguments are supplied, they extend and override *keywords*. Roughly equivalent to:

```py
def partial(func, /, *args, **keywords):
    def newfunc(*fargs, **fkeywords):
        newkeywords = {**keywords, **fkeywords}
        return func(*args, *fargs, **newkeywords)
    newfunc.func = func
    newfunc.args = args
    newfunc.keywords = keywords
    return newfunc
spice pecan
#

It basically freezes some arguments for your function

quasi hound
#

tysm

#

i've been tearing my hair out since last night over this

astral gazelle
#

alternatively you can shadow the variable in the lambda inputs

l = {msg: lambda msg=msg: print(msg.upper()) for msg in ['a', 'b', 'c']}
``` note the msg=msg
quasi hound
#

oh makes sense

spice pecan
#

Yeah, using default arguments forces active evaluation

quasi hound
#

alternatively if the arg is a string can i just do str(msg) inside the lambda somewhere to create a new string from it and not use a reference copy that way?

#

does the string constructor work like that

#

the partial thing is good but not the most readable

spice pecan
#

Not really, the issue here is that msg is not looked up until the dictionary is created

quasi hound
#

oh ok

spice pecan
#

The lambda is not aware of the value msg had at the point of its creation, all it knows is that it's supposed to look that name up and use whatever it gets in return

quasi hound
#

yeah my project works now

#

yall saved my ass

#

thanks again

forest lark
#

No idea which chat this question would fit in, but if I have a dict of products by id, and another dict of data pertaining to those products(quantity in cart, chosen warranties, etc.), would that second dict be considered "metadata" for the first dict?

pseudo cradle
#

Technically it's data about the data, which would make it metadata. However, I personally wouldn't use it because when I hear metadata I think of associated data to a file.

visual shadow
#

this question is probably fine for python general

pseudo cradle
#

Also that

#

So the other day we had a really interesting discussion about propagating nulls throughout code which I've tried to adopt into my own code, but I was curious as to how people handled propagating nulls when you have functions with multiple returns. For example:

return_1, return_2 = null_func()

Will throw an error because you're trying to unpack None. It gets pretty awkward to handle this because you have to do something like:

func_returns = null_func()
if func_returns:
  return_1, return_2 = func_returns

It would be nice to handle this at a decorator level instead of doing the above for every function, but not sure if that's possible.

halcyon trail
#

At that point you might just want to consider returning a dataclass/named tuple instead

terse orchid
#

In a task of a job application the gave me some instructions. In the document, this is being told.

They asking for a flask app. Do you think they want me to implement blueprints or not? I barely need two api functions.
"you can implement direct api functions" or "you definitely should implement a blueprint with flask.Blueprint"?

unknown.png
halcyon trail
#

So, as an example, suppose you have a function that simultaneously computes mean and stddev, you use it like pop_mean, pop_stddev = compute(), with a dataclass you'd just do pop_stats = compute() or something like that, and then just use pop_stats.mean and pop_stats.stddev. It even has advantages over multiple returns, with multiple returns it's up to the user to get the order correct when unpacking, and it's easy to make a mistake

#

those kinds of mistakes cannot happen with a dataclass

#

obviously you have to check for null either way, but with the dataclass version you never plan to unpack anyway, so you don't need to do this silly two step dance where you first assign to a single variable, then check null, and then unpack

visual shadow
#

but pop_stats = compute(), when being given a None, still would be none.

halcyon trail
#

Yes, but you don't need to unpack after

visual shadow
#

ah, i see what you mean

halcyon trail
#

Either way, if you get an Optional, you have to check for None

#

I thought you were just unhappy about having to first assign, and then unpack, when all you want to do is unpack

#

I've felt the same way before

#

Another nice thing about dataclasses vs returning a simple tuple particularly when you have optionals, is that with optional and tuple the type is already starting to become annoying to read

#

Optional[Tuple[float, float]] in my example above, and that's the simplest case

#

and it can keep getting worse, Optional[Tuple[float, bool, List[Foo]]] etc and things become unreadable so fast

#

This is obviously a general issue but it's just exacerbated here by the Optional

pseudo cradle
#

This is something I've toyed around with, actually. Considering making all the function returns in our libraries be uniform. Dancing back and forth between tuples and dicts

swift imp
#

Is there documentation on object anywhere? I just recently learned about __subclass__ which captures all direct subclasses of a class that inherit from object and I had no idea of its existence.

halcyon trail
#

tuples are kinda sub-optimal because the things don't hav enames, with dicts there are names but nothing helps you check you got the names right

#

named tuples avoid both of those issues. I'd personally just use a dataclass, but you can use a named tuple as well

paper echo
#

attrs

#

use attrs

#

or yeah dataclasses i guess

#

namedtuples if you absolutely must

halcyon trail
#

attrs supports slots, which would eliminate/mitigate one of the downsides of dataclass compared to tuple

deft pagoda
#

namedtuples are great --- they have mostly the same footprint as a tuple

halcyon trail
#

I'd be curious now to measure the memory usage of a named tuple, vs a tuple, vs a slots attr class, vs a regular dataclass (no slots)

spice pecan
#

Tuple VS namedtuple has no difference IIRC

peak spoke
#

Yeah they work through properties that access the tuple that's actually holding the data, technically there's some cost to the type itself but the biggest disadvantage is that the namedtuple type is somewhat costly to create (and its simplicity if it's not desired)

halcyon trail
#

it seems like slots is actually very efficient, closer to a tuple/namedtuple than a regular class

#

collections.namedtuple also kind of sucks, you define them using string literals which is kinda eww

#

it's nice to have the option for fancy programmatic stuff

#

but for simple things feels pretty silly

peak spoke
#

it does accept a list of field names, but I prefer NamedTuple if it makes sense to use it

halcyon trail
#

namedtuples also don't have type annotations if you don't use typing.NamedTuple, but I suspect that NamedTuple is basically identical to attrs with slots

#

except much more limited. So at that point, I don't really see a reason to use it ever.

peak spoke
#

it uses namedtuple from collections internally for the actual type, I find it very useful to group information

halcyon trail
#

Sure, you're declaring a class though, idk, if I knew how to measure it rigorously I would, but from what I can see e.g. from the linked post, I think slots implementations are going to have very similar per-object memory usage

#

attrs will let you group information just as well, and it's a lot less brittle, e.g. if you decide you want to allow mutation, you just unset frozen, with NamedTuple you have to change to something else

peak spoke
#

a namedtuple gets you some behaviour for free like the repr and a couple of methods (that you'll probably never use) compared to a class with slots but I'd expect the memory usage to be a bit higher. dataclasses, attrs etc. feel a bit overkill to me if all you're using it is to group and get named access to a structure you return for example

#

My most recent use of it was when I needed to accept/return a group and a name as an identifier but the order they should be in wasn't exactly clear, so I just used the namedtuple to get rid of ordered access

halcyon trail
#

namedtuple doesn't get rid of ordered access, that's part of the thing

#

it does get rid of unpacking though

#

sorry, my mistake, it gets rid of nothing

#

it's just a superset of tuple behavior

peak spoke
#

Yeah it's still there to be used for unpacking or something but the main interface moves from having to order the paremeters in some way to just accessing the names from the object

halcyon trail
#

dataclass doesn't support slots, so if you care enough about slots, I can see how in very specific contexts adding a dependency on attrs would be "overkill"

#

but, realistically if you already have attrs as a dependency, or if dataclass supported slots, I don't see what's overkill about it

#

named tuples actually generate more dunder methods etc, by default, since they're hashable, inequality comparable, iterable, indexable, "sized" (length works on them)

#

looks like NamedTuple is implemented using slots as well (that's what I suspected)

peak spoke
#

The only thing with slots it should be doing is setting them to be empty

deft pagoda
#

real programmers use cluegen

halcyon trail
#

yeah maybe you're right, I didn't see slots in the source, not sure what that comment was on about (I saw a github comment with someone saying it uses slots)

deft pagoda
#

faster than attrs and dataclasses

peak spoke
halcyon trail
#

Meh, really depends what you're doing, I'd actually prefer things to be initialized at startup than lazily

#

Also his "fireball" story isn't that compelling, attrs already solves this with kw_only init, which dataclasses is getting soon, and IMHO it should be the default anyway for most of your dataclasses/attrs

#

Really, at the end of the day, attrs, dataclasses, and cluegen, are all insane, even if python is dynamic, it should support a way to declare a class with fixed fields, of known type, first class, in the language, nicely, with reasonable perf 🤷‍♂️

deft pagoda
#

why add a dependency with 1000s of lines of code when you can just drop in a 100 lines

halcyon trail
#

why do you think the number of lines of code in a dependency matters?

deft pagoda
#

debugging

halcyon trail
#

I don't follow

#

I've never had a problem debugging through dataclasses

deft pagoda
#

that's cause there's too much code to go through, you'd follow if there was less code

halcyon trail
#

it's you that I don't follow, not the code 🙂

deft pagoda
#

i'm just following the code

halcyon trail
#

I've never needed to debug the internals of dataclasses or attrs, ever...

#

they do what it says on the tin

deft pagoda
#

i've never needed attrs

halcyon trail
#

more important than LOC of your dependency is how well supported it is, how many other people use it (and are running into bugs and reporting them, before you ever do), etc

#

I mean you needed functionality similar to it, i presume, otherwise why mention cluegen.

deft pagoda
#

i mentioned it because no one else did

halcyon trail
#

A: If you're using it, you do. You maintain cluegen.

deft pagoda
#

the best

halcyon trail
#

idk, it's incredibly common to need something like cluegen/attrs/dataclasses/pydantic, unless you like to work with unstructured data exclusively, which I guess some people do

#

maybe it would just be simpler to fork python, add first-class dataclass functionality, and maintain it myself 🤔

tidal marten
#

How do you go about making an enum with multiple values? Using namedtuple as the value for the enum?

#

I also seem to remember finding a way to subclass the Enum class itself, but cannot find the link to that

halcyon trail
#

If you Google python enum, you should probably get a hit inside the official docs

#

Or search the docs directly

#

Not exactly sure what you mean by multiple values for the enum

#

But yeah the enum values can be anything, including tuples

#

A frozen dataclass would be good

tidal marten
#

I mean like if I want

class MyEnumValue(NamedTuple):
    name: str
    value: Any

class MyEnum(Enum):
    SOME_VALUE = MyEnumType('name', 'value')
#

But, come to think of it, I don't need enum in my case

halcyon trail
#

Yeah you can do that

signal tide
#

Not sure if you could use a named tuple with it or not tho

languid adder
tidal marten
#

I ended up using data class since I don't really need to convert arbitrary value to enum

tidal marten
#

Is the only way to get class attribute is by using PEP 520?

halcyon trail
#

I'm kind of confused how you ended up using dataclass instead of enum, they do very different things, so I don't really understand

tidal marten
halcyon trail
#

okay

#

by "get class attributes", you mean, you want to iterate/reflect over the attributes of a class?

tidal marten
#

Yeps, but I want to get them in the order they are declared

#

That's why I reckon I don't need enum

halcyon trail
#

the best way to do that if I understand you correctly is to use fields

#

err, why do you reckon that?

#

these two things, again, have nothing to do with each other

tidal marten
#

Because I ended up putting my dataclasses inside a class and process them declaratively? That's why I need to fetch them in the order they are declared.

#

I was going to treat the dataclass as enum values but rekon it won't be necessary since I can just use the reference as some kind of enum anyway

raven ridge
#

you put your dataclasses in a class? That's... weird. Why not put them in a module like normal?

mild flax
#

not sure I understand, but if you need them in order, why not put them in a list?

tidal marten
#

I mean instantiations of my dataclass, lol

#

Idk, I'm just trying things out

sand goblet
#

Why doesn’t python have binary search tree?

modest willow
#

maybe some 3rd package supports bst?

flat gazelle
#

@sand goblet there is the bisect module which provides the core logic

#

Well, for array backed binary search trees

#

And well, you are generally supposed to use a set/dict

eager trail
#

I am majorly confused about logging
why does this print 1

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()
logger.info(1)

but this does nothing

logger = logging.getLogger()
logger.setLevel(logging.INFO)
logger.info(1)
gleaming rover
eager trail
#

ohhhh

#

thanks!

rugged merlin
#

hello, i don't know why it doesn't work. Plz help me.

unknown.png
unkempt rock
#

It should be if otvent == rand

#

Also, tab the else over to below the if