#internals-and-peps

1 messages · Page 33 of 1

quick snow
#

C extensions can incref the object and store its address wherever, right?

hollow swift
#

Hii everyone

slender hamlet
#

hey

grave jolt
#

You could do gc.get_referrers(obj) and then try to un-bind the references to objon each of them

#

And then you don't explicitly kill the object. So if you successfully killed all the references, it's going to die due to normal garbage collection. Otherwise, it keeps on living

#

like a best-effort thing

#

though that seems like a huge footgun with unclear benefits

flat gazelle
#

maybe global del could del the reference, and error out if the object still has references leftover. That should still let you have predictable lifetimes for objects where it's relevant.

feral island
uneven raptor
#

eventually we may get a protocol that identifies all references of an object for some concurrency-related things

night shale
#

Hi I need help

pure shard
#

yo all

#

konichiva

hybrid granite
#

hey

merry venture
#

if only this was possible

image.png
static hinge
# 
if case (_, _) = all_length:
vernal narwhal
glass mulch
#

Hi, does anyone with a JIT build want to help figuring out whether lafleur found a real JIT bug?

rain trellis
glass mulch
steel solstice
feral island
#

yeah that looks wrong, maybe open an issue on CPython? if it's a sphinx bug somebody will redirect it

static hinge
#

@~deco seems like some cool syntax.

feral island
#

it's meant to render without the module name

teal escarp
#

can someone help me ?

glass mulch
spark magnet
#

please no advertising

mint oracle
#

Alright

grave jolt
#

Why does ast.fix_missing_locations return the modified node, if it mutates it as well? Is that just a historical accident?

#

same with ast.copy_location and ast.increment_lineno

swift imp
#

Its an annoyance u have to do when modifying the ast or generating code from scratch with AST and go to dump it

feral island
faint river
faint river
#

3.13

>>> Union[int, str] | 'complex'
typing.Union[int, str, ForwardRef('complex')]

3.14

>>> Union[int, str] | 'complex'
Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    Union[int, str] | 'complex'
    ~~~~~~~~~~~~~~~~^~~~~~~~~~~
TypeError: unsupported operand type(s) for |: 'typing.Union' and 'str'
#

is this a bug, or is the change mentioned somewhere?

mild cobalt
faint river
feral island
white nexus
#

is it filed anywhere, or should I go report it?

#

(I'm assuming its already been filed tbh)

feral island
white nexus
#

done!

grave jolt
swift imp
echo gull
#

Can you give me some tips on the best way to learn Python?

grave jolt
#

the typical convention is to return None if you're mutating the argument, so I was surprised

swift imp
#

yeah I see now ur question, thank you 🙂 . I agree it is surprising

steel solstice
#

Maybe it's just easier to chain?

grave jolt
#

On discourse, does github issue status when previewing adjust to the UI language of the message author? Or am I tripping

image.png image.png
#

See opened on the left and відкрито on the right

feral island
#

sure looks like it

grave jolt
#

how does that even happen? o_O

#

is the preview handled on the client side?

feral island
#

must be I guess. when you write the comment some JS runs that prefetches it?

grave jolt
#

strange feature for sure

#

mostly for the 99% of readers who don't know what відкрито means

#

should I post about it on the meta topic?

merry venture
#

assuming that the comment is right saying that it's is_unionable that is the problem

#

oooh, it's not this code path

#

interesting.

feral island
#

working on a fix

merry venture
#

yeah, i see the problem. contrary to how casting is done in unionbuilder_add_single, we don't call type_check() on the _Py_union_type_or because we want to adhere to the operator protocol and return NotImplemented instead of raising an error that each arg must be a type

#

if it was any different, that would break the liskov principle, sort-of

in some parallel scheme of things, PEP 604 could have specified that | on types has its own rules though. that would perhaps simplify things a bit, but wouldn't allow good semantics with custom proxies that would want custom behavior on being |-ed with Unions. ...perhaps?

@feral island will you rewrite wrapping in ForwardRef to C or resort to Python's _type_convert?

feral island
# 
index c4ece0fe09f..e5342dd5faf 100644
--- a/Objects/unionobject.c
+++ b/Objects/unionobject.c
@@ -256,13 +256,8 @@ is_unionable(PyObject *obj)
 PyObject *
 _Py_union_type_or(PyObject* self, PyObject* other)
 {
-    if (!is_unionable(self) || !is_unionable(other)) {
-        Py_RETURN_NOTIMPLEMENTED;
-    }
-
     unionbuilder ub;
-    // unchecked because we already checked is_unionable()
-    if (!unionbuilder_init(&ub, false)) {
+    if (!unionbuilder_init(&ub, true)) {
         return NULL;
     }
     if (!unionbuilder_add_single(&ub, self) ||
``` is what I have but haven't tested yet
merry venture
#

oh so that would probably introduce new semantics and never return NOT_IMPLEMENTED

#

because we're resorting to _type_convert and not really reinterpreting the error to NOT_IMPLEMENTED

#

or am i mistaken

feral island
#

yeah I need to think through the implications more but this would restore 3.13 behavior and allow some other things

merry venture
#

i'll check

feral island
#

in particular int | str | "float" would become legal

#

but not int | "float" which is somewhat dubious

merry venture
# feral island yeah I need to think through the implications more but this would restore 3.13 b...

python 3.13 was very loose, yeah

❯ ./python
Python 3.13.9+ (heads/3.13:d7473f7a476, Oct 20 2025, 21:01:02) [GCC 15.2.1 20250808 (Red Hat 15.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from typing import Union
>>> Union['str', 4] | object() | type('', (), {})
typing.Union[ForwardRef('str'), 4, <object object at 0x7f9b75f09c30>, __main__.]
>>> Union['str', object(), type('', (), {})]
typing.Union[ForwardRef('str'), <object object at 0x7f9b75f09c90>, __main__.]
#

but it still does check whether the types are conceptually correctly placed

>>> Union['str', 4] | object() | type('', (), {}) | ClassVar[int]
Traceback (most recent call last):
    ...
TypeError: typing.ClassVar[int] is not valid as type argument
#

TIL!

#

well i guess i meant type('', (), {})() and not type('', (), {}) which is a valid class by all means in terms of runtime

feral island
#

yeah that's typing._type_check, it just rejects a couple of special things

merry venture
#

definitely +1 on restoring the old behavior i guess

#

runtime processing engines have the most to care about here

merry venture
#

the proposed change would be the one compatible with the pep ^

#

no... not too fast

#

the pep just doesn't seem to specify this case

feral island
#

that PEP is extremely low on detail

merry venture
#

yeah

feral island
#

I'm still sort of surprised that was enough back in 3.10 days

merry venture
#

i hope nobody relied on return-type dispatching

feral island
#

left a comment

merry venture
feral island
#

oh the issue also already discusses the same thing (the issue is not just with return types, but more generally we don't necessarily look at the right annotation)

merry venture
#

yeah but my thing doesn't find the first one, just discards the obvious return annotation

feral island
#

I think the right approach is to first look at inspect.signature to find the right parameter name to use, then grab that one from the annotations dictionary

#

or I guess straight from the signature

merry venture
#

found a different edge case as well

>>> @foo.register
... def x(*args: Unpack[tuple[str, int]], **kwargs):
...     print("x called")
...     
Traceback (most recent call last):
  File "<python-input-9>", line 1, in <module>
    @foo.register
     ^^^^^^^^^^^^
  File "/home/bswck/Python/cpython/Lib/functools.py", line 913, in register
    raise TypeError(
    ...<2 lines>...
    )
TypeError: Invalid annotation for 'args'. typing.Unpack[tuple[str, int]] is not a class.
#

this should perhaps find str...

#

but that's very deep in the woods

#

at this point you can just pass str

#

and we can report that special forms aren't supported or something

#

unless Annotated? are Annotated types supported at all?

#

oh ok they seem to be

feral island
merry venture
#

nice.

feral island
merry venture
#

not sure about *args: T though

#

because args can be a ()

#

therefore i think we should raise on a variadic param being registered

#

but again

#

edge case.

#

nooo that would break things

#

and someone might have a valid use case, and in their code *args could always be 1+ in size just by convention. let's not touch that

#

i'm seeing skip_bound_arg in _signature_from_callable. it's not public API but definitely of use here?

#

no, not much. at this point it's always a function, not a bound method. :/

#

so yeah, inspect.signature would be great primarily as a source of truth

feral island
#

hm yes, maybe there isn't a simple way to recognize the self argument

merry venture
#

the problem lies in an edge case of custom descriptors, right
something registered to singledispatchmethod must be a callable belonging to the class
plain function is obvious, classmethod is obvious, and if we specialcase staticmethod to understand the first param as non-self that solves 99% of problematic cases

the rest is an open question

#

perhaps we can just change the error message to recommend using verbose type API (.register(cls)) if the descriptor is ambiguous?

#

oh, the thing is more complicated in singledispatchmethod

feral island
#

weird inconsistency in the error messages, is this worth fixing?

>>> class T:
...     def __init__(self, a, b): pass
...     
>>> T(a=1, **{"a": 2, "b": 3})
Traceback (most recent call last):
  File "<python-input-4>", line 1, in <module>
    T(a=1, **{"a": 2, "b": 3})
    ~^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: __main__.T() got multiple values for keyword argument 'a'
>>> T(c=3)
Traceback (most recent call last):
  File "<python-input-5>", line 1, in <module>
    T(c=3)
    ~^^^^^
TypeError: T.__init__() got an unexpected keyword argument 'c'
#

maybe it's because the multiple kwargs thing gets raised earlier

merry venture
# feral island weird inconsistency in the error messages, is this worth fixing? ``` >>> class T...

i'd fix it because the latter is more helpful if both __init__ and __new__ are present and perhaps not uniform:

>>> class T:
...     def __init__(self, **kwargs): pass
...     def __new__(cls, *, a, b): return object.__new__(cls)
...     
>>> T(a=1, **{"a": 2, "b": 3})
Traceback (most recent call last):
  File "<python-input-11>", line 1, in <module>
    T(a=1, **{"a": 2, "b": 3})
    ~^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: __main__.T() got multiple values for keyword argument 'a'
>>> T(c=3)
Traceback (most recent call last):
  File "<python-input-12>", line 1, in <module>
    T(c=3)
    ~^^^^^
TypeError: T.__new__() got an unexpected keyword argument 'c'
#

unless in that specific case there is no access to what exactly is the target

#

maybe .T() is just type.__call__?

#

it's in ceval so it can't be fixed? we don't know what T will be

fallen slateBOT
#

Python/ceval.c lines 1725 to 1732

kw_found:
    if (PyStackRef_AsPyObjectBorrow(localsplus[j]) != NULL) {
        _PyErr_Format(tstate, PyExc_TypeError,
                    "%U() got multiple values for argument '%S'",
                  func->func_qualname, keyword);
        goto kw_fail;
    }
    localsplus[j] = value_stackref;```
merry venture
#

T can be anything

#

nvm i think this can't be fixed

feral island
#

calls with more than one of the same kwarg are always invalid

merry venture
#

yeah

feral island
#

maybe the weird thing is that T() is mentioned at all

merry venture
#

yes!

#

that shouldn't be there

# 
>>> (lambda: 1/0)(a=1, **{"a": 2, "b": 3})
Traceback (most recent call last):
  File "<python-input-14>", line 1, in <module>
    (lambda: 1/0)(a=1, **{"a": 2, "b": 3})
    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: __main__.<lambda>() got multiple values for keyword argument 'a'
#

but perhaps it can be helpful because it's resolved at the time of call

#

so we know what we tried to call... although the problem isn't calling what, it's calling how

#

target may be a very weird and unrelated thing if it's improperly wrapped

#

and that can be confusing

#

the whole focus should be on the call itself

merry venture
# feral island maybe the weird thing is that T() is mentioned at all

nice.

>>> SimpleNamespace(__qualname__=1 .__truediv__)(a=1, **{"a": 2})
Traceback (most recent call last):
  File "<python-input-23>", line 1, in <module>
    SimpleNamespace(__qualname__=1 .__truediv__)(a=1, **{"a": 2})
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
TypeError: <method-wrapper '__truediv__' of int object at 0x00007FFFA373D3A8>() got multiple values for keyword argument 'a'
#

probably should be as close as possible to

>>> whatever(a=1, a=2)
  File "<python-input-24>", line 1
    whatever(a=1, a=2)
                  ^^^
SyntaxError: keyword argument repeated: a
#

but not a syntax error, of course

feral island
#

hm it also looks like the callable has no qualname it raises a KeyError?

#

no

#

the exception from qualname replaces the TypeError though ```>>> class X:
... def getattr(self, attr):
... if attr == "qualname": raise ZeroDivisionError
... raise AttributeError(attr)
...

X()(a=1, **{"a": 2})
Traceback (most recent call last):
File "<python-input-6>", line 1, in <module>
X()(a=1, **{"a": 2})

~~~^^^^^^^^^^^^^^^^^

File "<python-input-5>", line 3, in getattr
if attr == "qualname": raise ZeroDivisionError
^^^^^^^^^^^^^^^^^^^^^^^
ZeroDivisionError

merry venture
#

smart! __qualname__ could only have been patched this way -- not through class assignment, because that one is eagerly validated

merry venture
#

breaks if __str__ is incorrect as well:

>>> type("", (), {"__str__": lambda _: 1})()(a=1, **{"a": 2})
Traceback (most recent call last):
  File "<python-input-50>", line 1, in <module>
    type("", (), {"__str__": lambda _: 1})()(a=1, **{"a": 2})
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
TypeError: __str__ returned non-string (type int)
#

or, to make it less obvious

# 
>>> mysterious(a=1, **{"a": 2})
Traceback (most recent call last):
  File "<python-input-52>", line 1, in <module>
    mysterious(a=1, **{"a": 2})
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^
TypeError: __str__ returned non-string (type int)
#

i think that, in principle, raising an exception in ceval should be free of side effects

merry venture
merry venture
#

(the last one overlaps with where the call is correct but the signature is incompatible)

#

while __qualname__ isn't likely to fail, __str__ is much more likely to overtake exceptions, particularly if the instance they operate on is partially initialized (AttributeErrors)

#

but i think it's rare for an object to be only partially initialized if it's being called.

#

and still, that is only relevant for objects that implement __call__ on their own. most of the time we're simply calling functions and classes.

white elbow
#

waht is this channel about

fierce coral
#

Description is in the top ribbon on the discord display

merry venture
fallen slateBOT
#

Lib/test/test_capi/test_float.py line 205

quiet = int(not signaling)```
radiant garden
#

it is clear!

limpid forum
#

If signaling was a bool, then I'd agree it's clear... But it's 0 or 1, not bool. not non_boolean_here is not clearer

merry venture
#

well booleans are ints so
it kinda makes sense

forest pawn
#

Searching around the Python forum in Ideas didn't lead me to anywhere.... anyone know if there's ever been a proposal/discussion around slicing dictionaries by keys? Basically if I have {"a": 1, "b": 2, "c": 3} and, say, ["a", "b"] , to get {"a": 1, "b": 2} without resorting to a comprehension call. Feels like there should be a method on dicts for this

# 
>>> dict_a = {"a": 1, "b": 2, "c": 3}
>>> dict_b = {"c": 4, "d": 5}
>>> list(dict_a.keys() | dict_b.keys())
['a', 'c', 'b', 'd']
>>> list(dict_a.items() | dict_b.items())
[('d', 5), ('b', 2), ('c', 4), ('a', 1), ('c', 3)]
>>> dict_a | dict_b
{'a': 1, 'b': 2, 'c': 4, 'd': 5}
>>> dict_a - dict_b
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for -: 'dict' and 'dict'
>>> dict_a.keys() | dict_b.keys()
{'a', 'c', 'b', 'd'}
>>> dict_a.keys() - dict_b.keys()
{'a', 'b'}

like we're so close to something interesting here IMO! I don't ... I don't think - is really a great operator in this case for this but I do think a method would make a lot of sense

#

also in terms of del... like wiping a set of keys in one go

#

(I might just bem issing a method on dict, been staring at the docs and not seeing the thing)

sour thistle
#

just to mention, dict.items() supports -, but as an operation over a set of (key, value) tuples, so it only removes if both the key and value are the same

>>> x = {'a': 1, 'b': 2, 'c': 3, 'd': 4}
>>> y = {'c': -1, 'd': 4}
>>> dict(x.items() - y.items())
{'a': 1, 'c': 3, 'b': 2}

actual slicing (via the slicing syntax) wouldn't work since dictionaries can store slices as of 3.12 though, dict[:] = ... will literally store that slice object (slice(None, None, None) in this case) as a key

limpid forum
#

To add:
my_dict["a", "b"] is just indexing using a tuple and tuples are valid keys

(because it's a comma that makes a tuple, not parentheses - those are only needed in empty tuple and when comma can has a different meaning, e.g. in function calls or when defining a dict literal with a tuple key)

indigo hull
#

Curios if anyone has any advice or an opinion on this-

A while back someone filed a github issue to update asyncio's CancelledError to inherit from BaseException so that it doesn't get caught by generic try/except loops. At first it was rejected, but the Python maintainers came around and updated the inheritance in 3.8.

There's another class named CancelledError in Python from concurrent.futures. Both represent a cancelled task, so it makes sense to me that both would inherit from BaseException. I'm wondering if this was just a miss when they updated asyncio's CancelledError, or if this was intentional. I'm considering trying to make a similar PR to cPython, but it would be my first one.

uneven raptor
#

did that inheriting from Exception cause an actual bug for you, or is this speculation?

#

I’d expect the places where concurrent.futures.CancelledError can be raised to be pretty different than where asyncio.CancelledError can be raised

forest pawn
merry venture
#

(likely because of __rsub__ because you can't subtract valuesview from valuesview)

indigo hull
clear hill
#

Is it a bug in CPython that running round(2**31, -2**31) seems to never complete? what even does it mean to round to a negative number of digits?

feral island
#

from memory, it errors out, but spends time proportional to the size of the negative number or something

clear hill
#

interestingly I’m also coming at this as fallout from devdanzin’s fuzzing campaign

sour thistle
clear hill
#

maybe that should be documented in the round docs. Right now it just says negative ndigits is accepted but doesn’t explain what it does. Thanks for experimenting!

clear hill
uneven raptor
feral island
#

yes this cannot really be changed in a backwards-compatible way

#

maybe it's worth the disruption but 🤷‍♀️

clear hill
feral island
#

doesn't really affect your fix though

clear hill
#

I was talking a little loosely

feral island
#

fair

clear hill
#

rephrased a little more precisely

feral island
#

looks good to me. disappointing that numpy won't support 2**16-bit floats though 😛

clear hill
#

that’s a problem for numpy of the fuuuuutuuuuure

feral island
#

that's how we get a new y2k problem

grave jolt
#

iirc overflow of signed ints is undefined in C/C++, while overflow of unsigned ints is two's complement, or something like that

grave notch
#

Here's a doozy: https://github.com/jburgy/skean/pull/2 is my (currently unsuccessful) attempt at upgrading a PEP-523 hack past 3.10. The PyFrameObject/_PyInterpreterFrame split which happened in 3.11 broke most of the things and I am under the grand delusion that I fixed all but one. Who's foolish enough to help me with that last issue (accessing locals of the current _PyInterpreterFrame not the closest "complete" PyFrameObject)?

GitHub
Upgrade to 3.11 by jburgy · Pull Request #2 · jburgy/skean

python/cpython#27077 broke this thoroughly by changing the _PyFrameEvalFunction signature. I did what I could to resurrect it but am now at an impasse. I need what used to be called f_localsplus ...

raven ridge
# grave notch Here's a doozy: https://github.com/jburgy/skean/pull/2 is my (currently unsucces...

I don't follow what you mean by "complete"... You say:

PyEval_GetFrame and PyThreadState_GetFrame only return complete frames. The relevant frame is, by definition, incomplete since we're in the middle of evaluating it!

But by that definition, every frame on the stack is incomplete... every frame on the stack is still being evaluated - that's literally what it means for a frame to be on the stack, that it will eventually be returned to and continue executing

#

All of which is to say, PyEval_GetFrame can certainly return a frame that's still being evaluated

grave notch
#

My statement comes from observing that PyFrame_GetLocals(PyThreadState_GetFrame(tstate))) and PyCode_GetVarnames(PyUnstable_InterpreterFrame_GetCode(iframe)) don't line up during my test. The former contains fibonacci while the latter contains n which leads me to believe PyThreadState_GetFrame(tstate) represents the caller and iframe the callee. This jives with the following bit of cpython:

/* Determine whether a frame is incomplete.
 * A frame is incomplete if it is part way through
 * creating cell objects or a generator or coroutine.
 *
 * Frames on the frame stack are incomplete until the
 * first RESUME instruction.
 * Frames owned by a generator are always complete.
 *
 * NOTE: We allow racy accesses to the instruction pointer
 * from other threads for sys._current_frames() and similar APIs.
 */
static inline bool _Py_NO_SANITIZE_THREAD
_PyFrame_IsIncomplete(_PyInterpreterFrame *frame)
{
    if (frame->owner >= FRAME_OWNED_BY_INTERPRETER) {
        return true;
    }
    return frame->owner != FRAME_OWNED_BY_GENERATOR &&
           frame->instr_ptr < _PyFrame_GetBytecode(frame) +
                                  _PyFrame_GetCode(frame)->_co_firsttraceable;
}

static inline _PyInterpreterFrame *
_PyFrame_GetFirstComplete(_PyInterpreterFrame *frame)
{
    while (frame && _PyFrame_IsIncomplete(frame)) {
        frame = frame->previous;
    }
    return frame;
}

static inline _PyInterpreterFrame *
_PyThreadState_GetFrame(PyThreadState *tstate)
{
    return _PyFrame_GetFirstComplete(tstate->current_frame);
}
#

PyEval_GetFrame calls _PyEval_GetFrame which, in turns, calls _PyThreadState_GetFrame. Only difference is you don't need to pass a thread state, it will use _PyThreadState_GET

#

Even PyFrameObject *_PyFrame_GetFrameObject(_PyInterpreterFrame *frame) (which I can't access because it's static inline) asserts that !_PyFrame_IsIncomplete(frame) before returning _PyFrame_MakeAndSetFrameObject so that wouldn't help either. I believe I need to address the interpreter frame's locals directly but don't know how

grave notch
#

Oh, I just found something completely evil and disgusting to access .localplus: <PyObject **>((<char *>_frame) + 72). (72 because, in 3.12, _PyInterpreterFrame contains 8 pointers, an int (stacktop), an unsigned short (return_offset), a char (owner), and padding to the next double word before localsplus)

grave notch
#

The hack above can be made marginally less disgusting if you squint hard enough and remember struct padding and flexible array members in C: (<PyObject **>_frame) + 9. Forget that _PyInterpreterFrame is a struct with an array of PyObject * inline at the end. Think of it as an array of PyObject * where the first few (9 to be specific in 3.12 and 3.13) entries represent a header. Then skip over that header.

clear hill
halcyon trail
#

I also don't even really understand how that can work, there's no guarantee that value is hashable at all, so if this is the case, it can only work quadratically

#

unless I'm missing something

radiant garden
#

it only supports dict_items objects is how it manages

halcyon trail
#

sorry, don't think I followed that

#

the - operation is performed "over a set of (key, value)" items

radiant garden
#

Yeah, and you are given the (key,value) pairs from a source that ensures that the key is hashable

halcyon trail
#

but not the value

sour thistle
# 
>>> a = [1, 2, 3]
>>> d = {'x': a}
>>> d.items() - d.items()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'list'
>>> d = {'x': tuple(a)}
>>> d.items() - d.items()
set()
radiant garden
#

Oh interesting!

halcyon trail
#

okay, erroring is at least better than silently using quadratic behavior, thanks for pointing that out

#

I definitely still think having a - operator here is a mistake though, personally

weak hawk
# halcyon trail but not the value

you don't need the value to be hashable
i.e. dict_items is not really a set of tuples as far as I know, it (internally) supports O(1) access by key

radiant garden
#

I thought it used the keys' hash and then compared values only on key collision, this is definitely funny behavior

halcyon trail
#

and as we can see, in that case you do need the value to be hashable

weak hawk
#

oh, to produce the set

#

but you don't need it for the comparison/finding I think

sour thistle
halcyon trail
#

anyhow - back to the operator existing to start with
I suspect if you told 100 python devs who didn't already know that d1.items() - d2.items() was valid and asked them to guess the behavior

#

a huge fraction if not the majority would get it wrong

#

and it's rather niche to start with - IMHO would be better if it didn't exist

radiant garden
#

I guess I just find the set-like behavior a unexpected

halcyon trail
#

well, there's two semi-reasonable behaviors, and both are "set-like"

#

but yeah, I expected subtraction just on key

#

I mean I actually expected subtraction to just not work at all

#

I always saw the point of .items() as producing something for iteration, random iterable things don't generally support subtraction

halcyon trail
#

i.e. d1.items() == d2.items() will run, evenif the values are not hashable

#

which makes me think it's silently degrading to N^2 behavior

weak hawk
halcyon trail
#

.items() is supposed to behave like a set of (key, value)
but a set of (key, value) will always do == in O(min(N1, N2))

weak hawk
#

the comparison for the value will take time... but a dict_items object can retrieve the (k, v) pair for a given k in O(1) time

halcyon trail
#

hmm I guess you're right actually

somber temple
#

Why i have this error plz: Traceback (most recent call last): File "C:\Program Files (x86)\Thonny\lib\site-packages\thonny\plugins\cpython_frontend\cp_front.py", line 39, in init self._send_msg(ToplevelCommand("get_environment_info")) File "C:\Program Files (x86)\Thonny\lib\site-packages\thonny\running.py", line 1157, in _send_msg self._proc.stdin.write(serialize_message(msg) + "\n") OSError: [Errno 22] Invalid argument
Process ended with exit code 0

#

@devout estuary

static hinge
somber temple
#

oh

#

sry

white nexus
#

So I've noticed that python 3.8-3.13 had signed git tags for their releases, but starting with python 3.14 and continuing into python 3.15, there aren't signatures on the git tags or other information confirming that 3.14 and 3.15 are signed by who they are said to be signed by

crisp locust
#

PSF made a terrible decision oh well

crimson hatch
#

3.14 and 3.15 have a new RM (Hugo), so it could just be that he doesn't have commit/tag signing set up

white nexus
crimson hatch
tribal estuary
#

helo

spark magnet
crisp locust
spark magnet
crisp locust
spark magnet
crisp locust
spark magnet
worldly venture
crisp locust
spark magnet
crisp locust
worldly venture
#

(that is why the Steering Council exists)

spark magnet
crisp locust
#

My focus is the Technology Stack nothing else

spark magnet
crisp locust
#

Anyhow the decision was made so be it

spark magnet
#

@crisp locust btw, we also have yet to see that this decision harms the tech stack. People are stepping up to support the PSF because of this decision.

raven ridge
#

I'm out of the loop - what decision are we debating the merits of?

pallid tapir
radiant garden
#

it makes sense given that the PSF's explicit goal is to foster the community

quick snow
#

I think even independent of the PSF's goals and values this decision was a no-brainer, given that the terms were so overreaching, applied to all activity of the PSF and would have allowed the NSF to unilaterally claw back money that was already sent, if they decided the PSF didn't do enough (against its own stated values).
It's hard to think of a PSF decision in the last years that should be less controversial.

spark magnet
#

disappointing to have people on the sidelines bemoaning the decision but not willing to actually look at the hard choices.

raven ridge
#

Yeah, seems like a no-brainer to me. If they took the money, would they have to stop having the Charlas track at PyCon, for instance? Would they need to stop providing travel grants? Would they need to stop funding PyLadies? I think the answer to all of those is "yes"

spark magnet
#

Mostly it comes down to the uncertainty about what the US administration would consider a violation of the law. They'd rug-pull the money and the PSF would have no recourse except to sue the federal govt, which would be costly, time-consuming, and unlikely to result in the money being returned soon enough to matter.

#

And looking at the current administration, I think they would consider many activities like you've mentioned to be violations.

raven ridge
#

Heck, the government could argue that the stenographers captioning the talks at PyCon are a form of DEI

winged sphinx
radiant garden
#

The live feed has secret messages hidden in the pixels!

spark magnet
#

sure, that's where the brainwashing happens....

boreal umbra
#

I think it's unlikely that the administration would try to claw back the funds unless Trump personally had a reason to want to punish the PSF for the spectacle. But I think the PSF is wise not to open themselves up to that risk

white nexus
#

I am so glad psf didn't take it

white nexus
boreal umbra
#

I work for a federally funded company. Trump is a monster, but there are (at least for now) people in government trying to get things done.

white nexus
#

I don't have faith in the US government anymore but that's not really related to this channel typically, aside from it being related to the topic at hand

simple sand
radiant garden
#

this is probably getting off topic

gilded ember
#

Hello

analog musk
#

Hey everyone, just read through the thread — really interesting points from both sides. I get where Brother Richard is coming from about focusing on the tech, but I also think the PSF made the right call here. If funding comes with strings that could limit inclusion or independence, that’s not worth the risk. Better to keep the community and the language aligned than compromise for short-term money.

frigid bison
#

what is the use of co_nlocals? Isn't it just len(co_varnames)?

feral island
frigid bison
#

co_varnames should only contain locals

gilded flare
gilded flare
crimson hatch
# frigid bison what is the use of `co_nlocals`? Isn't it just `len(co_varnames)`?

co_varnames is only populated lazily for backwards compatibility. On free-threaded, co_varnames is kept in a thread-local cache (_PyCoCached), and only populated lazily from localsplus (which are locals, cell vars aka variables referenced in nested scopes, and free vars aka variables defined in an outer scope). co_localsplusnames is used instead of co_varnames in the core interpreter loop e.g. https://github.com/python/cpython/blob/07912f86323dc5a13f4d070b7e3f2c3cb2850d8b/Python/ceval.c#L1656

fallen slateBOT
#

Python/ceval.c line 1656

co_varnames = ((PyTupleObject *)(co->co_localsplusnames))->ob_item;```
maiden kayak
frigid bison
crimson hatch
#

Unfortunately I don't know the history there, I'm only familiar with the more recent interpreter loop implementation. I would recommend looking at history of codeobject.c for clues if no one else replies

quick snow
#

Yep, both co_nlocals and co_varnames were introduced in the same commit (884afd65) in 1995; that's Python 1.x days.

frigid bison
clear hill
maiden kayak
clear hill
#

in the past, political and oversight consideration have prevented governments from abusing that too hard - now we have an executive that doesn’t care about that stuff

plush narwhal
#

like the PSF refusing the 1.5M

spark magnet
spark magnet
plush narwhal
#

someone on the board i think

spark magnet
plush narwhal
#

i saw it in my RSS feed

crisp locust
boreal umbra
spark magnet
boreal umbra
#

US federal law is often written in a vague way that gives the executive leeway to apply the law however they want. And then people who disagree with the executive's application challenge it all the way to the supreme court, who rule in favor of the administration if they were appointed by a member of the administration's political party, else against.

spark magnet
#

tbh, all laws are vague to differing degrees, which is why we have juries and judges, appeals, etc.

boreal umbra
#

Yes, but US laws are especially so, because the threshold to pass laws is unusually high (60 votes in the most lopsided legislative chamber on earth) and the two parties are especially uncooperative

spark magnet
crisp locust
spark magnet
#

what position do you find interesting?

crisp locust
# spark magnet what position do you find interesting?

We were forced to withdraw our application and turn down the funding, thanks to new language that was added to the agreement requiring us to affirm that we "do not, and will not during the term of this financial assistance award, operate any programs that advance or promote DEI, or discriminatory equity ideology in violation of Federal anti-discrimination laws."

spark magnet
boreal umbra
#

"you can't promote diversity, equity, and inclusion; you can't discriminate"

These are ostensibly exact opposites.

crisp locust
# spark magnet do you understand how subjective laws can be? In this case, the funding isn't ev...

Statement by the government

“The federal government must ensure that taxpayer money is used lawfully and for the public good,” said Assistant Attorney General Harmeet K. Dhillon. “The very foundation of our anti-discrimination laws rests on the principle that every American deserves equal opportunity, regardless of race, color, national origin, sex, religion, or other protected characteristics.”

spark magnet
crisp locust
#

Federal antidiscrimination laws prohibit discrimination on the basis of protected characteristics, including race, color, religion, sex, and national origin. The U.S. Supreme Court has consistently held that policies or practices based upon protected characteristics are subject to rigorous judicial scrutiny. Race-based classifications are subject to strict scrutiny, requiring a compelling governmental interest and narrowly tailored means to achieve that interest. Sex-based classifications are subject to heightened scrutiny, requiring an exceedingly persuasive justification
and substantial relation to an important governmental objective. Discrimination based on other protected characteristics, such as religion, is also evaluated under analogous standards. Entities receiving federal funds must comply with applicable civil rights laws, including:

Title VI of the Civil Rights Act of 1964: Prohibits discrimination based on race, color, or national origin in any program or activity receiving federal financial assistance. This includes most educational institutions, healthcare providers, and state and local government agencies.

Title VII of the Civil Rights Act of 1964: Prohibits employment discrimination based on, or motivated by, race, color, religion, sex, or national origin, in any terms, conditions, or privileges of employment, including hiring, promotion, demotion, termination, compensation, job transfers, training, or access to employment privileges and benefits.

Title IX of the Education Amendments of 1972: Prohibits discrimination based on sex in education programs or activities receiving federal financial assistance. Title IX protections extend beyond athletics and include addressing sexual harassment, sex-based harassment, admissions policies, and equal access to resources and programs.

spark magnet
crisp locust
#

Nothing subjective about the law

spark magnet
crisp locust
#

I can provide real life examples unlawful practices

spark magnet
crisp locust
spark magnet
#

do you understand that the funding isn't subject to conviction of breaking a law, but merely an official claiming a law was broken?

spark magnet
crisp locust
spark magnet
#

@crisp locust do you think those were 1500 prosecutions and convictions?

crisp locust
spark magnet
crisp locust
spark magnet
#

and laws are not clear-cut.

crisp locust
spark magnet
crisp locust
spark magnet
crisp locust
spark magnet
crisp locust
spark magnet
#

I think the current administration would say that emails sent only to women would be a violation of the law.

crisp locust
#

We are never going to agree. Have a nice day.

spark magnet
#

again, bowing out. got it.

plush narwhal
#

I don't understand this:

#

"do not, and will not during the term of this financial assistance award, operate any programs that advance or promote DEI, or discriminatory equity ideology in violation of Federal anti-discrimination laws."

spark magnet
plush narwhal
#

and how could the NSF even say this?

#

like "stop promoting equality" for 1.5M

spark magnet
plush narwhal
spark magnet
plush narwhal
#

so why would they tell someone to break the law?

#

or not promote equality

spark magnet
plush narwhal
#

to not promote "discriminatory equity ideology"

#

or is it just saying don't promote it, but dont be racist

spark magnet
#

i'm not sure what you are uncertain about.

plush narwhal
spark magnet
#

they will say that diversity programs are fundamentally discriminatory. that's where the subjectivity comes in.

plush narwhal
spark magnet
clear hill
static hinge
#

Who has the backport?

grave jolt
#

oh, one of the steering council members was the PEP author

#

I wonder what happens if a PEP is co-authored by 4/5 members or 5/5 members?

#

PEP 798 also got accepted

#

!pep 798

fallen slateBOT
static hinge
#

^ with a single change, no yield from

quick snow
#

No chain.from_iterable

feral island
gilded flare
swift imp
#

Gotta be like one of the fastest peps I've seen accepted

faint river
#

ayyy, two peps in one day?

#

I will have to read them. They sound interesting

boreal umbra
#

what a glorious day

faint river
#

ok lazy imports are really good

#

I like the point that is made about typing-only imports

#

actually, can anyone confirm whether a type statement will reify a lazy import?

feral island
#

however, using a lazy import in a field annotation in a dataclass will reify it

faint river
#

that makes sense. Assuming that is because the dataclass evaluates the annotation for purposes of generating the class init?

feral island
#

I feel like we might need to think about a way to address that

#

it's primarily for figuring out whether they're ClassVars

#

or InitVars

faint river
grave jolt
#

Why is that needed? I thought you can't put ClassVar in a type statement

#

(from the type system perspective)

faint river
grave jolt
#

Oh, you're talking about not reifying dataclass attributes

faint river
#

The problem is that they must reify to check whether they are a typing.ClassVar or not.
Checking whether they are a dataclasses.InitVaris a non-issue IMO because are using dataclass there anyways

feral island
#

In the PEP 810 demo at https://lazy-import-demo.pages.dev/ you can put this example to see what happens

import sys
from dataclasses import dataclass

lazy import lazy_demo.utils

@dataclass
class Foo:
    x: lazy_demo.utils.T

print("After lazy imports:")
print(f"  'lazy_demo.heavy' loaded? {'lazy_demo.heavy' in sys.modules}")
print(f"  'lazy_demo.utils' loaded? {'lazy_demo.utils' in sys.modules}")
print()
Python Lazy Imports - Interactive Demo

Python Lazy Imports - Interactive WASM Demo

faint river
#

is it appropriate to try and change the implementation of dataclasses such that it does not evaluate the annotations?

#

I don't see any attributions to any company in it (unlike abc), so I would assume so? I am not versed in contributing to cpython

spark magnet
faint river
#

yes, not me specifically, but yes

spark magnet
#

are we talking in two channels about this?

faint river
#

what

spark magnet
faint river
#

python discussion was just "hey people look, cool new thing"

spark magnet
#

it sounds like not evaluating annotations would be a big change, but maybe I'm missing something.

faint river
#

you would need to parse the string representation of an annotation to see whether it is a typing.ClassVar or dataclasses.InitVar.
As far as I have been thinking, that would require that dataclasses.dataclass analyzes the code that is is imported into.
This is because it is possible to define those names yourself and not be the actual types that the dataclass decorator needs to check for.
Sounds bad.

feral island
#

I sort of feel like we should try to fix this but can't think of a very elegant way to do it. You could look for whether ClassVar appears in the string version of the annotation or something but that's arguably not enough, users could alias it

faint river
feral island
faint river
#

what's the non-general answer

feral island
#

though whether the contribution will be accepted is always uncertain

grave jolt
#

A cursed fact about ClassVar: normally ClassVar[Final[X]] and Final[ClassVar[X]] are not allowed in a class. Unless it's a dataclass, in which case it's the only way to create a final classvar (both orders are possible)

#

So whether x: Final[int] = ... defines a classvar depends on whether it's a dataclass or not

feral island
#

only difference might be that some modules could have an owner with a strong sense of ownership

uneven raptor
#

good lucking contributing to something like antigravity, lol

faint river
feral island
#

no, a core developer

#

companies don't own anything in Python

faint river
feral island
#

there might be some random notices on some modules but that's just for historical reasons

uneven raptor
#

that means "abstract base class", not google's parent company

feral island
#

they're referring to "# Copyright 2007 Google, Inc. All Rights Reserved."

uneven raptor
#

oh

feral island
#

though I wonder how much of that module's code is still what Google contributed

uneven raptor
#

I wonder if that comment is compatible with cpython's current license

spark magnet
#

@faint river contribution can be difficult, but it's never because some company owns a module.

feral island
faint river
#

I wonder if the license is technically revokable

feral island
#

I hope not, would be very impractical if it was

spark magnet
faint river
#

splendid

crimson hatch
# spark magnet the PSF owns the IP of the code.

well, technically I don't think this is true. The contributor agreement grants the PSF an irrevecobale, perpetual license to redistribute contributions under any license approved by the PSF. However e.g. here there is wording:

PSF understands and agrees that Contributor retains copyright in its Contributions

faint river
#

!pep 798 question on this

uneven raptor
#

someone should give emma her core team role

fallen slateBOT
crimson hatch
faint river
# 
[(*it if cond else it) for it in its]

is something like this (or similar, with a syntax chance) possible? Essentially, conditional flattening

spark magnet
feral island
crimson hatch
faint river
# 
>>> [x if x else *y]
  File "<stdin>", line 1
    [x if x else *y]
                 ^
SyntaxError: cannot unpack only part of a conditional expression
feral island
white nexus
#

!pep 810

fallen slateBOT
white nexus
#

the lazy imports method can only be set once, it would be nice if it could be set for a module such as setuptools and therefore all imports within the setuptools namespace would use that filter method

eg sys.set_lazy_filter_namespaced('setuptools', method)

meager nacelle
#

dataclasses not evaluating is a tricky one, there's already some extra fiddling for string annotations recognising ClassVar and InitVar. It actually looks like the type comparison for string annotations as it is now will be broken if ClassVar is lazily imported 🤔

#

Yes, dataclasses pulls from the module dict directly so it retrieves the lazy import object.

#

So combining lazy import typing with from __future__ import annotations and a typing.ClassVar annotation doesn't work correctly with @dataclass.

faint river
meager nacelle
#

No, it still switches things to string annotations

faint river
#

Oh hmm.

#

So it's best to not do that import for 3.14+ code then

meager nacelle
# 
from __future__ import annotations

from dataclasses import dataclass
lazy import typing

@dataclass
class X:
    a: typing.ClassVar[str] = "H2G2"
    b: int = 42

print(X())

The ClassVar annotation has no effect currently due to how the comparison is implemented.

feral island
#

I think it does a string match

meager nacelle
#

I just tried it

#

with the current lazy build

#

it checks for "typing" in sys.modules

#

which it isn't

feral island
#

oh lol

#

I tried in the REPL which I guess imports typing somewhere

meager nacelle
#

yeah the new repl imports half the stdlib

#

old repl might also import typing, can't remember

feral island
#

I don't think so, that one wasn't written in Python

meager nacelle
#

The check isn't for a string match either it's more fiddly than that

#

Guess this comment isn't true any more

    # If typing has not been imported, then it's impossible for any
    # annotation to be a ClassVar.  So, only look for ClassVar if
    # typing has been imported by any module (not necessarily cls's
    # module).
feral island
#

That's a good catch

meager nacelle
#

(I'll note that for my own dataclass-like implementation I do just check for the string ClassVar and document it as such)

feral island
#

yes I feel like that would make life easier

meager nacelle
#

Too late to change that now though

feral island
#

I suppose compatibility makes it hard for us in CPython to change yes

#

I don't know whether anyone is doing shenanigans like having a utility module that does from typing import ClassVar as MyFancyName and then use that in a dataclass

meager nacelle
#

But I saw the hoops dataclasses was jumping through and decided against it 😆

#

I've heard people might use from typing import ClassVar as CV, but I'm not sure I've seen that actually used anywhere other than to test that the aliases work for dataclasses.

feral island
#

actually from typing import ClassVar as _ClassVar is more common

meager nacelle
#

Technically that one should still work

grave jolt
#

and also maybe the string ClassVar as (to detect multiline imports)

meager nacelle
#

Either way it's not a guarantee we can remove

white nexus
#

the lazy imports method can only be set once, it would be nice if it could be set for a module such as setuptools and therefore all imports within the setuptools namespace would use that filter method

eg sys.set_lazy_filter_namespaced('setuptools', method)

Where can I suggest/raise this?

#

Anyone know of an existing thread? I haven't used dpo

boreal umbra
#

(carry on)

white nexus
#

ok I was looking into removing the documentation for curses.__version__ because it doesn't exist and hasn't since at least 3.8

#

but now I'm trying to determine why that is

fallen slateBOT
#

Modules/_cursesmodule.c lines 5447 to 5451

rc = PyDict_SetItemString(module_dict, "__version__", curses_version);
Py_CLEAR(curses_version);
if (rc < 0) {
    return -1;
}```
white nexus
#

that doesn't add up

#

the rc must be derefed at some point before that line is run, hm

feral island
white nexus
#

huh

#

oh my god

#

its because its not in __all__

#

when did that change, then.... thanks jelle

feral island
white nexus
#

right yeah I meant that but went to the other part

#

its underscored, and because __all__ doesn't exist/contain it, it doesn't exist in curses

#

!d curses.version

fallen slateBOT
white nexus
#

python/cpython#140606

neon troutBOT
feral island
#

I sort of suspect this has always been broken

white nexus
#

oh same probably

feral island
#

not motivated enough to build a Python 2 to check though 🙂

white nexus
#

I think the proper way to do this is to remove the docstring and the _curses.__version__ string and then remove just the docs for a backport in a seperate pr

#

this would be my first pull to python 🙂

feral island
#

that sounds right

white nexus
#

is there a devguide page on making pull requests?

feral island
#

not sure. it's gh-<number>: Descriptive title where number is the issue number

#

so in this case something like Remove _curses.__version__

white nexus
# neon trout

should this be reopened since its not quite a duplicate?

#

or keep the tracking to the other issue about version attributes

feral island
#

seems close enough to keep the same issue

white nexus
#

fair

#

and now this pr requires c code changes which I don't know, heck

feral island
#

it's just removing some lines, right? shouldn't be too hard

white nexus
#

this diff seems wrong in terms of memory management

image.png
feral island
#

oh yeah keep the Py_CLEAR

white nexus
#

PyCLEAR(curses_version) should probably be called here if I'm reading this right

#

and then I should probably be able to remove Py_DECREF

feral island
#

otherwise it's leaked in that error path

white nexus
#

No no like this

    rc = PyDict_SetItemString(module_dict, "version", curses_version);
    Py_CLEAR(curses_version);
    if (rc < 0) {
        return -1;
    }
feral island
#

oh yeah that works

#

should that (non-underscored version) also be deprecated?

white nexus
#

What's the difference between Py_CLEAR/Py_DECREF| where could I find that?

white nexus
#

not the code here

feral island
#

I see

#

it's a bytes object which sort of suggests nobody since the Python 3 change has cared about it

feral island
white nexus
feral island
#

this is almost certainly documented in C API docs, if you google like site:docs.python.org Py_CLEAR you should be able to find it

white nexus
#

I can likely safely remove __version__, or do I need to deprecate the definition in _ncurses

#

it would only be usable with _curses.__version__ and has never been documented as such

#

I guess I just make the pull and put that in the body

feral island
#

yeah since it's only on the private module I feel it's OK to remove, another core dev might want a deprecation cycle though

white nexus
#

I'll do it with no deprecation but someone can ask for it in which case I'll figure out how

#

thanks jelle!

raven ridge
#

!d Py_CLEAR

fallen slateBOT
#
Py\_CLEAR 

void Py_CLEAR(PyObject *o)```
Release a [strong reference](https://docs.python.org/3/glossary.html#term-strong-reference) for object *o*. The object may be `NULL`, in which case the macro has no effect; otherwise the effect is the same as for [`Py_DECREF()`](https://docs.python.org/3/c-api/refcounting.html#c.Py_DECREF), except that the argument is also set to `NULL`. The warning for [`Py_DECREF()`](https://docs.python.org/3/c-api/refcounting.html#c.Py_DECREF) does not apply with respect to the object passed because the macro carefully uses a temporary variable and sets the argument to `NULL` before releasing the reference.

It is a good idea to use this macro whenever releasing a reference to an object that might be traversed during garbage collection.

Changed in version 3.12: The macro argument is now only evaluated once. If the argument has side effects, these are no longer duplicated.
white nexus
#

I think I did it right python/cpython#141052

neon troutBOT
raven ridge
#

You can use backticks to avoid turning double underscores into bold text:

`curses.__version__`
#

And you'll need to click that "sign CLA" button

white nexus
#

already signed, and thanks for the description update

plush narwhal
#

python/cpython#141052

neon troutBOT
white nexus
#

yay its merged python/cpython#141052

neon troutBOT
white nexus
#

thank you, @limpid forum, @feral island, and @raven ridge, for your help with this change!

feral island
#

congrats!

astral gazelle
#

<@&831776746206265384> crossposting ad

quick snow
#

!kick 495204443323760661 You've just been told not to spam

fallen slateBOT
#

:incoming_envelope: :ok_hand: applied kick to @glad aurora permanently.

grave jolt
#

Is there some centralized list of __dunder__ methods? Right now some of them are on the Data Model page but some (like __replace__) are not

white nexus
#

There's a replace method?!

#

!d object.replace

fallen slateBOT
#
object.\_\_replace\_\_ 

object.__replace__(self, /, **changes)```
This method should create a new object of the same type, replacing fields with values from *changes*.

Added in version 3.13.
white nexus
#

Huh!

white nexus
grave jolt
white nexus
#

Can probably throw something together shortly as a bash script

faint river
fallen slateBOT
#
copy.replace 

copy.replace(obj, /, **changes)```
Creates a new object of the same type as *obj*, replacing fields with values from *changes*.

Added in version 3.13.
grave jolt
#

yeah, the destroyer of covariant frozen datclasses

white nexus
#

Why do I forget what covariant means again

merry venture
merry venture
# white nexus Why do I forget what covariant means again

a covariant type G means that if TSub is a subtype of T, then G[TSub] is a subtype of G[T].

this takes place for immutable containers, for instance a Sequence[bool] is assignable to Sequence[int], whereas a mutable list[bool] is not assignable to list[int] due to invariance.

merry venture
#

!e ```py
import hello, traceback
traceback._print_exception_bltin = lambda exc, file=None, /: hello.main()

1 / 0

fallen slateBOT
merry venture
#

investigating linecache line availability when source files are deleted

fallen slateBOT
#

:x: Your 3.14 eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "/home/main.py", line 7, in <module>
003 |     runpy.run_path(str(p))
004 |     ~~~~~~~~~~~~~~^^^^^^^^
005 |   File "<frozen runpy>", line 287, in run_path
006 |   File "<frozen runpy>", line 98, in _run_module_code
007 |   File "<frozen runpy>", line 88, in _run_code
008 |   File "t.py", line 1, in <module>
009 | ZeroDivisionError: division by zero
merry venture
#

k i'll use ci runners... it's platform-specific things

deep dirge
merry venture
#

or do you mean there were opinions that it's unnecessary to fix those typos

atomic current
deep dirge
merry venture
merry venture
#

@feral island does typeshed have checks to discover stdlib modules that aren't present? or is the process organic, like, someone comes and opens up a PR to add a new module?

for stdlib there's both pure and non-pure modules, so it seems best to validate our stubs against sys.stdlib_module_names of a particular version.

#

i think i'll just add this to that new project of specialized runtime checks of typeshed.
let me know if there are any other runtime checks typeshed could use

merry venture
#

adding an issue to track

merry venture
feral island
merry venture
feral island
#

I think I made an issue years ago with all missing standard library modules

#

There's a few left perhaps but it tends to be internal stuff that probably shouldn't be in typeshed anyway

merry bramble
#

Yeah, i think typeshed is pretty complete these days for public, documented modules

feral island
frigid bison
fallen slateBOT
#

Python/bytecodes.c line 197

inst(LOAD_FAST_AND_CLEAR, (-- value)) {```
frigid bison
#

one way I think I could emulate it is doing LOAD_FAST_CHECK, catching a possible UnboundLocalError triggered if the local var doesn't exist yet and push NULL

#

but that seems like it would impact performance a lot + I would have to keep the NULL behaviour. I'd like to emulate whatever this opcode is used for without using NULL as it does not exist in 3.10

feral island
#

I'm not sure what you're trying to do with the bytecode so hard to say how you should handle it

frigid bison
#

I'm trying to "compile" 3.12 bytecode to different versions, but specifically 3.10 and 3.11.

frigid bison
#

back into a separate function

#

but that seems hard to recognize

#

I don't have a good enough mental model of how the fast storage works, especially as it changed between versions. As far as I understand they need this opcode to keep the locals of the comprehension separate from the locals of the code the comprehension is in? Can you confirm that?

feral island
frigid bison
# feral island yes, that's right

How does this opcode achieve that though? It's still using the same "table" of locals, it just loads a possibly NULL value and clears it afterwards. How does this keep it separate?

#

thanks for indulging my weird question btw

feral island
frigid bison
#

how does it know which one to use? do you have any documentation or pull requests on that?

#

I've noticed documentation of internals like this is rather sparse. The pull requests usually have the hidden gem information 😄

feral island
#

Yeah it's all going to be in the source. This is all intentionally undocumented implementation details

#

Looks like what happens is that when we enter the comprehension, we stash the outer local of the same name on the stack

#

Then put it back when the comprehension is done

# 
...     x = 1
...     y = [x for x in [2]]
...     return x * y
...     
>>> dis.dis(f)
   1           RESUME                   0

   2           LOAD_SMALL_INT           1
               STORE_FAST               0 (x)

   3           LOAD_CONST               1 ((2,))
               GET_ITER
               LOAD_FAST_AND_CLEAR      0 (x)
               SWAP                     2
       L1:     BUILD_LIST               0
               SWAP                     2
       L2:     FOR_ITER                 4 (to L3)
               STORE_FAST_LOAD_FAST     0 (x, x)
               LIST_APPEND              2
               JUMP_BACKWARD            6 (to L2)
       L3:     END_FOR
               POP_ITER
       L4:     STORE_FAST               1 (y)
               STORE_FAST               0 (x)

   4           LOAD_FAST_CHECK          0 (x)
               LOAD_FAST_BORROW         1 (y)
               BINARY_OP                5 (*)
               RETURN_VALUE

  --   L5:     SWAP                     2
               POP_TOP

   3           SWAP                     2
               STORE_FAST               0 (x)
               RERAISE                  0
ExceptionTable:
  L1 to L4 -> L5 [2]
frigid bison
#

so technically that means this inlining could also be supported in 3.10

feral island
#

yeah I'm not actually sure anything would go badly wrong if you just replace LOAD_FAST_AND_CLEAR with LOAD_FAST

#

maybe the worst thing is that some reference gets kept alive too long?

frigid bison
#

the worst thing that could happen is that it panics on the assert

#

of value != NULL

frigid bison
feral island
frigid bison
fallen slateBOT
#

Python/bytecodes.c line 193

assert(value != NULL);```
frigid bison
#

in 3.11 and 3.10 it throws the UnboundLocalError

#

okay okay I finally understand how to handle this

#

it just pushes NULL to the stack to save it after the comprehension, but if I do some logic then I can avoid re-assigning x if it never existed before the comprehension

merry venture
#

is it welcomed that external contributors fix them?

spark magnet
merry venture
# spark magnet where do you mean?

a couple in https://github.com/python/cpython/blob/3ce2d57b2f02030353af314d89c5f6215d2f5c96/InternalDocs/interpreter.md#L36-L43, perhaps more

  1. unclosed lparen from line 36 in line 37
  2. a link without href in line 40 ([`_PyEval_EvalFrameDefault()`]) -- judging by https://devguide.python.org/internals/interpreter/, i'm assuming that that document isn't rendered anywhere else than github, so there's no crossref magic and the link just misses a href
  3. some links point to wrong places: [`_PyEval_EvalFrame()`](https://docs.python.org/3.14/c-api/veryhigh.html#c.PyEval_EvalCode) -- i don't think _PyEval_EvalFrame is equivalent to PyEval_EvalCode -- there's public PyEval_EvalFrame below it: https://docs.python.org/3.14/c-api/veryhigh.html#c.PyEval_EvalFrame, and these things seem to do very different things in the code
fallen slateBOT
#

InternalDocs/interpreter.md lines 36 to 43

(this is used in the implementation of
[`​gen.throw`​](https://docs.python.org/3.14/reference/expressions.html#generator.throw).

By default, [`​_PyEval_EvalFrame()`​](https://docs.python.org/3.14/c-api/veryhigh.html#c.PyEval_EvalCode)
simply calls [`​_PyEval_EvalFrameDefault()`​] to execute the frame. However, as per
[`​PEP 523`​](https://peps.python.org/pep-0523/) this is configurable by setting
`​interp->eval_frame`​. In the following, we describe the default function,
`​_PyEval_EvalFrameDefault()`​.```
merry venture
#

so just asking if one or a bunch of PRs to fix these are welcome

spark magnet
merry venture
#

thanks!

spark magnet
#

if you make a PR and get resistance, ping me on it.

merry venture
deep dirge
deep dirge
uneven raptor
#

you can fix refs if you want, I don't have much of a preference there

clear hill
grave jolt
#

!pep 814

fallen slateBOT
grave jolt
#

New pep just dropped

#

Seems like we're not going to get a HAMT in the stdlib after all 😔

quick snow
#

I thought we have a HAMT in the stdlib, it's just not exposed to Python land

#

(PEP 603 is not rejected though)

swift imp
#

Yeah but it's been inactive since 2019

#

Idk how anyone can consider it

clear hill
#

rpds-py is a thing…

#

IMO python needs a first-class way of marking variables as mutable a la rust but frozen types will help too

boreal umbra
clear hill
boreal umbra
clear hill
#

I’m officially an old

halcyon trail
#

how would that actually work though once you start aliasing things

#

this sort of thing is just kind of messy in GC languages, which is why you don't really see it much

#

the closest thing I can think of is D, but that's not exactly a super popular language, and I don't remember all the details of their system

clear hill
#

makes sense, the other side is that in the type system you really want the mutability to show up in the type, so things get messy if types can go through transitions from mutable to immutable - this showed up in the discussion around freeze

halcyon trail
#

the lower hanging fruit is just to use a lot more frozen or at least read only types. but that's kind of hard because python has really committed itself to dict, list, and set

#

a good example of a language that does this without making things very elaborate is Kotlin.
in Kotlin when you do the equivalent of a list comprehension, you get List, not MutableList, as the result.

#

so, in general there's just a lot less mutation floating around in a Kotlin program than in a python program.

clear hill
#

don’t have to tell me that… all the mutable cython cdef classes that need passes to work on the free-threaded build without allowing new kinds of thread safety issues is kind of daunting

halcyon trail
#

hah, yeah

#

I wasted hours or maybe even days a couple years ago because I made the mistake of having a dataclass with with a nested member and didn't use default_factory

#

this is back when datacalsses allowed that

#

some_member: Foo = Foo()

#

and then the program was using a thread pool or something; bad things ensued

spark verge
#

I thought dataclasses did that also

meager nacelle
#

I think it does now but didn't always?

halcyon trail
#

correct

#

I don't know in which exact version they started doing it

#

maybe 3.11 or so

#

I fixed that particular bug when it happened of course but at some point later, we upgraded python version and we had to fix this in a couple dozen other places

#

I knew to use default_factory for list/dict/set but it didn't really occur to me for nested dataclasses for whatever reason

#

that defining a type with some defaulted members is so error prone and unintuitive in python is definitely not one of the bright spots

meager nacelle
#

I don't actually do the check for this in my own implementation. I'm not really happy with the use of hashability as a proxy as it can be wrong in both directions.

#

That said it's probably right more often than not.

halcyon trail
#

yeah, it's not an ideal situation

#

it's just generally a problem though in several contexts

clear hill
#

don’t look at how numpy dtypes are hashed

halcyon trail
#

you have the same problem in dicts in general

clear hill
#

it’s bad

halcyon trail
#

and not just in python, most GC languages have this issue

#

mostly they each try to do something that heuristically prevents the most common issues

clear hill
#

this is not good but mostly it doesn't matter in practice

#

you might have bad times if you make dtypes the keys of a dictionary along with strings

halcyon trail
#

oof

#

Yeah

#

I don't think I've ever mixed them. I can definitely imagine a dict with dtypes as keys, and probably even written it at some point. but not a mixture.

meager nacelle
#

Is that not also something about how equality is defined, let alone hashing?

clear hill
#

it’s both, because equality and hashing are related

meager nacelle
#

Isn't it that two equal items should have the same hash but that two items with the same has aren't necessarily equal? I'm just surprised the equality check ignores the type.

clear hill
#

there's a ton of code that implicitly relies on this behavior so changing it is an enormous headache

#

lol:

>>> np.dtype(None) == 'float64'
True
#

because the default dtype is float64

meager nacelle
#

oof

#

Things you wish you didn't know

proven knoll
#

I'm Chinese, nice to chat with you.

merry venture
#

and you can reach it through _testinternalcapi

grave jolt
#

!e

s = str(encoding='utf-8', errors='strict')
assert s == ""
``` This is weird... why is `str` callable with just these kwargs?
fallen slateBOT
spark magnet
grave jolt
#

yeah I know

gilded flare
#

!d str

fallen slateBOT
#
str 

class str(*, encoding='utf-8', errors='strict')``````py

class str(object)``````py

class str(object, encoding, errors='strict')```
Return a [string](https://docs.python.org/3/library/stdtypes.html#textseq) version of *object*. If *object* is not provided, returns the empty string. Otherwise, the behavior of `str()` depends on whether *encoding* or *errors* is given, as follows.

If neither *encoding* nor *errors* is given, `str(object)` returns [`type(object).__str__(object)`](https://docs.python.org/3/reference/datamodel.html#object.__str__), which is the “informal” or nicely printable string representation of *object*. For string objects, this is the string itself. If *object* does not have a [`__str__()`](https://docs.python.org/3/reference/datamodel.html#object.__str__) method, then [`str()`](https://docs.python.org/3/library/stdtypes.html#str) falls back to returning [`repr(object)`](https://docs.python.org/3/library/functions.html#repr).
grave jolt
gilded flare
#

oh

#

okay i get it

clear hill
#

https://discuss.python.org/t/pre-pep-rust-for-cpython/104906

Discussions on Python.org
Pre-PEP: Rust for CPython

Introduction We (@emmatyping, @eclips4) propose introducing the Rust programming language to CPython. Rust will initially only be allowed for writing optional extension modules, but eventually will become a required dependency of CPython and allowed to be used throughout the CPython code base. Motivation Rust has established itself as a popular...

#

RiiR 😉

#

I kid I kid, that’s not what’s proposed

raven ridge
#

As someone who maintains the Python ecosystem at a large company (but speaking for myself, not for the company) I really hope this doesn't happen. We don't have a great story for ownership and maintenance of the Rust toolchain right now, and it'll be a big extra burden on my team if we need to ensure Rust is available and usable internally as a prerequisite for Python being available and usable internally

#

I'd much rather see CPython start using C++ than start using Rust. C++ is much safer than C, and everyone who has a C11 toolchain can easily install a C++11 toolchain too

crimson hatch
#

Well, it will be optional for at least a couple of years, so hopefully that would be time enough to develop a deployment plan? Would be very interested in your thoughts!

raven ridge
#

If it's optional but required for building some of the built-in modules, that's really not optional

#

It's not an option for us to ship an interpreter where some of the built-in modules don't work, even if they're newly added modules

crimson hatch
#

I should add a section about C++. It may be safer than C, but large C++ projects like Chrome and Firefox are still seeing significant memory safety improvements with Rust over C++

swift imp
#

Doesn't seem right

raven ridge
#

It wouldn't be the end of the world for us, we would definitely manage, but it would be much more disruptive for us than the change to start requiring C99 and then C11 were, and much more work for us to catch up and stay caught up

crimson hatch
crimson hatch
raven ridge
#

(though C++ would probably come almost for free 😉)

#

you can point gcc at a .cpp file and it does the right thing. I'm pretty sure that's true for clang as well. Not sure about MSVC, but if it does that too, that's all 3 of the main platforms...

clear hill
#

or polars or…

raven ridge
#

we use the upstream wheels

clear hill
#

but you do your own Python builds?

raven ridge
#

yep. we build the interpreter from source, but don't currently build 3rd party libs from source.

clear hill
#

makes sense why it hasn’t been a big pain until now

raven ridge
#

and we build the interpreter from source in a company-wide integration build environment that is, to put it mildly, challenging to keep up to date with new toolchains

clear hill
#

or i guess not now but maybe the future

#

this is all a big backdoor campaign to get rust compilers available in BigCo build environments 😉

raven ridge
#

you can't upgrade gcc without fixing everyone's janky old code that won't build with the new gcc. I shudder to think we'd wind up in the same situation for Rust, especially given that my team likely doesn't have the Rust expertise to shepherd through one of those upgrades.

crimson hatch
#

FWIW I expect Python would choose a rather conservative version of Rust to build with if integrated

#

I hope that would make your life a bit easier?

raven ridge
#

sure, better than the bleeding edge. but it doesn't really solve the problem around ownership and maintainership. It just kicks the can down the road for us. Eventually you'll want to upgrade to a newer Rust, and we'll need to have figured out some way to get rustc upgraded by then

clear hill
raven ridge
grave jolt
#

apparently Guido is on board

crimson hatch
#

(though it seems you have your own requirements for such things, and if so, sorry :/)

clear hill
#

@crimson hatch if I can intrigue you into working on https://github.com/mesonbuild/meson-python/issues/721 that would unblock a lot of interesting possible Rust work in scientific packages. I know that at one point there was a discussion about using meson/muon instead of autotools in the cpython build.

GitHub
Rust extension given invalid name by meson-python · Issue #721 · ...

Link to reproducible example: https://github.com/medley56/python_rust When using meson-python to package a Rust extension, I get the following error when running pip install . ../meson.build:10:14:...

crimson hatch
#

Yes, I actually was originally planning on working on re-designing the build system before introducing Rust but decided it was too big of a task to take on right now

raven ridge
crimson hatch
raven ridge
#

ah. ok, I stand corrected on that one. But what I said would apply for "upgrading" any of the existing optimization modules to Rust

crimson hatch
#

yes, and I think for 3.15 we probably won't do that

raven ridge
#

ok, I think you're right that if it is only brand new modules, we could probably kick the can down the road a little further.

#

as long as the modules are new, and there is a pure-Python fallback for them

round path
raven ridge
#

(I've just been saying "3.15" without paying any particular attention to what version this is proposed for inclusion in, heh)

crimson hatch
raven ridge
# crimson hatch (though it seems you have your own requirements for such things, and if so, sorr...

At the risk of over-explaining, I think the two "extra" requirements we've got are to not cause any performance regressions by upgrading (so no swapping an optimized 3.N module with an unoptimized 3.N+1 module), and to ensure that the interpreter we ship provides all of the publicly documented modules (thus the requirement for a pure-Python fallback for any Rust module that we might skip building)

#

we probably have some wiggle room on the latter, if necessary, but it'd put us in an awkward situation to say "Yes, we know that the release notes talk about the great new foobar module, but you can't use it at our company."

crimson hatch
wanton flame
crimson hatch
#

Though I'd prefer to never do that again 😅

clear hill
#

@crimson hatch where should I look in your PoC for the bindgen bindings? I’m curious how awful it would be to use it instead of pyo3-ffi in pyo3’s internals? or at least expose something that works like pyo3-ffi. I’m not particularly enthused about going through the CPython headers by hand for 3.15

#

I’m also a little curious how you’re handling some API details that the pyo3 ffi tests handle in hacky ways when they use bindgen

crimson hatch
#

The bindgen bindings are generated at compile time. Also DM incoming :)

#

But yeah the bindings will end up in target/[debug,release]/deps/cpython-sys-<hash>/out/c_api.rs

clear hill
#

thanks!

hybrid relic
#

Rust in cpython now, Rust is everywhere :P

#

(I'm probably going to cry at what will become of the build system once that happens, this probably spells the end of my attempt to help the MinGW-w64 team with "porting" the Interpreter given I can never seem to get the hang of Rust)

#

Unless... Any talk of improving the build system that I've missed lately?

crimson hatch
#

Prior to this work, I was hoping to improve the build system as a pre-requisite to Rust, but I felt that was an even more impossible change with a lot more work, so I decided to shelve it

spare willow
#

can anyone help me?

#

I am learning python language. I am very beginner in it. I am having the problem with terminal portion part. I can't able to run the python file from there and even can't ble to install Pip.

#

Please help.

primal cypress
#

I think where most people can agree on is to use rust in extensions, but not the core, like the interpreter

#

which makes sense

#

It's a smaller/'safer' step. Like if a mistake is made in this change, it won't be that bad since it's mostly just an extension, but if a bug was introduced into the interpreter because of rust, then people would definitely point fingers at this change, even though it's not necessarily because of it, if ykw I mean

winged sphinx
eager hollow
#

I have a high performance app making heavy use of memoryview objects. I was excited to discover that Python 3.14 added an index method to memoryview; however, when testing this, I found it's much slower to use memoryview.index() than it is to first convert the memoryview to bytes, and then call bytes.index(). I don't suppose anyone has also noticed this and/or might know why?

eager hollow
#

The line in question is:

idx = self.data.index(NEW_LINE, beg, len(self.data))

My overall program is 4.5x slower when leaving self.data as a memoryview instead of converting it to a bytes. This is very counter intuitive!

gilded flare
eager hollow
#

I would expect memchr to be used in this case.

gilded flare
eager hollow
#

Hmm... something is going on. I just tried creating a small reproducible example, and I'm not seeing any timing differences now. But in my real code, simply changing:

self.data = self.buffer[: partial_length + num_read]

to

self.data = bytes(self.buffer[: partial_length + num_read])

provides a 4.5x speedup. It's just counter intuitive that copying the memoryview to a bytes makes it much faster :) There must be some strange interaction in my program.

#

Got it !!!

import time


def main(obj):
    data = bytes(obj)  # removing bytes() => 860x slower !!!
    for _ in range(10_000):
        assert data.index(10, 100) == len(data) - 1


if __name__ == '__main__':
    data = memoryview(bytearray(64 * 1024))
    data[-1] = 10
    start = time.perf_counter()
    main(data)
    stop = time.perf_counter()
    print(f'time = {stop-start:.3f}')
eager hollow
hybrid relic
hybrid relic
clear hill
#

it works on Python 3.13 and older but on 3.14 somehow the signature ends up corrupted

grave jolt
#

I was trying to figure out why I can't get this function to work. Seems like it assumes that the function has at least one decorator applied (with @ specifically) for some reason

clear hill
grave jolt
#

that str(type(_)) thing was cursed yeah

clear hill
grave jolt
#

I'd expect py @foo def bar(): ... to be the same as ```py
def bar(): ...
bar = foo(bar)

clear hill
#

me too! I guess no one has noticed the limitation you found until now

grave jolt
#

Yeah, it's not a big deal I suppose

clear hill
#

this project is a reverse dependency of vLLM - my current project is to get vLLM running on the free-threaded build

grave jolt
#

I'm not sure why the function needs to strip decorators tbh

#

It also won't work if the first decorator spans multiple lines```py
@decorator_factory(
param1=1, param2=2)
@strip_multiline_string_indents
def banana():
...

#

since it literally removes the first line of source code, not the first decorator through the AST

clear hill
#

neither do i! but the principle of chesterton's fence applies to basically everything I'm doing lately...

grave jolt
#

sorry if I'm piling on with this function, I just don't like this kind of hasty meta-programming

clear hill
#

Thankfully I'm not responsible. This is not the craziest I've seen on this project. There was a unit test that was basically doing:

for i in range(sys.getrefcount(None)):
    ...

And no one noticed that in Python 3.12 the test started taking 15 minutes to complete instead of being near-instant

grave jolt
#

because of immortals?

clear hill
#

yeah, the immortal refcount is several billion

#

in 3.14 it's not 2^32 -1 anymore but it's large

grave jolt
#

who needs f-strings anyway

clear hill
#

f-strings for LLMs

#

i'm still not totally sure what this library does

crimson hatch
feral island
#

so that loop might be finding the wrong one

#

if that's true I feel I've done something good for the world by breaking that function

clear hill
#

TIL context variable tokens are also context managers in 3.14, no need to define a boilerplate decorator anymore

boreal umbra
#

What are those?

clear hill
#

@boreal umbra you can think of a context variable as a souped-up thread-local that is “context-local”. You can use them to store state in a context manager.

#

before python 3.14 you had to write:

import contextvars
import contextlib

v = contextvars.ContextVar("test", default='default')


@contextlib.contextmanager
def test_context(value):
    token = v.set(value)
    try:
        yield
    finally:
        v.reset(token)

print(v.get())

with test_context("context variable has a value!"):
    print(v.get())
grand river
clear hill
#

now you can write:

import contextvars

v = contextvars.ContextVar("test", default='default')

print(v.get())

with v.set("context variable has a value!"):
    print(v.get())
grand river
#

man it is really hard to do something right

#

look at my web extension

#

like i dont know how to market it lol

clear hill
#

@grand river that's not what this channel is for

grand river
#

would u like check it out?

raven ridge
fallen slateBOT
#
Rules

6. Do not post unapproved advertising.

merry venture
#

thanks for sharing

static hinge
clear hill
proper moss
#

hello, anyone interested in internals of GIL removal

primal cypress
#

definitely cool but the JIT rn is more interesting to me

quick snow
#

!clban 1417610459980234862 only here to advertise

fallen slateBOT
#

:incoming_envelope: :ok_hand: applied ban to @high kite permanently.

proper moss
primal cypress
proper moss
primal cypress
# proper moss where

my own benchmarks on my local machine (might not be as accurate)
There's this too https://doesjitgobrrr.com/ and
https://raw.githubusercontent.com/facebookexperimental/free-threading-benchmarking/refs/heads/main/results/bm-20251108-3.15.0a1%2B-7e2bc1d-JIT/bm-20251108-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-7e2bc1d-vs-base.svg
-# It's in code block cuz discord messes up the link lol

image.png
proper moss
primal cypress
#

yeah it's very clean

uncut quest
clear hill
#

i don’t think it was a meaningful test but it got extra un-meaningful with 3.12

rancid reef
#

Anyone helo me pls dm me

spark magnet
rancid reef
#

@spark magnet ok i am new

spark magnet
rancid reef
#

@spark magnet sir can I ask u thing

spark magnet
hybrid relic
#

Was digging through the build system's code and found a rather amusing comment that when the GIL is disabled the entire JIT is thrown into the trashcan 😆

primal cypress
#

it's not thread safe

hybrid relic
#

But I am curious though how does simply disabling the GIL result in the entire JIT not being used at all? I'd get if the JIT doesn't work properly with the GIL, but I don't think I've ever seen a system just mysteriously decide to silently not do anything at all if another is disabled/enabled

primal cypress
#

JIT has dynamic code generation and ref coutning and a lot of stuff

hybrid relic
primal cypress
hybrid relic
#

I did see that issue, what I got from it was that the compiler just straight up doesn't get used

#

(Normally you'd expect the compiler to fail spectacularly on free threading, not silently just check out and call it a day, which is a little amusing to me)

clear hill
clear hill
#

numba is also working on supporting the free-threaded build

#

you’re right that the whole topic is fraught - populating global caches in a scalable and thread-safe manner is tricky and a JIT is a pile of global caches

hybrid relic
#

I see

#

I wonder if the second Interpreter (Forgot the name) is affected by this as well, I do recall that it uses the same configure option as the JIT does

primal cypress
#

python has a lot of interpreters for different compilers, but do you mean tail-calling?

hybrid relic
#

It has like 2 tiers of Interpreters from what I remember? Not referring to tail calls or computed goto, I think the lower tier one was the profiling one and the higher tier one is supposed to be faster?

#

I really forgot everything about Python's internals, jeez

primal cypress
#

Tier 2 is the JIT I think tier one is specialising interpreter I think?

#

hmm

hybrid relic
#

I very vaguely recall Python had 3 executors planned when the project to make it faster was started, the baseline interpreter (This was the one that can select between switch, computed goto and tail calls I think), a faster interpreter and the JIT being the fastest

#

Hold on lemme check the docs

clear hill
#

are you thinking of subinterpreters?

#

i think each interpreter has its own JIT but not 100% on that

hybrid relic
#

No not really

#

I'm referring to the one implemented in executor_cases.h I believe

round path
#

That's the tier 2 interpreter. In reality we use it only for debugging the JIT. It's much slower than the base interpreter right now.

primal cypress
#

hmm I always thought tier 2 was the JIT, when you set the flag to enable the JIT, tier 2 is used? not sure if im getting this right

round path
#

the tier2 macro has multiple levels you can also pass --enable-experimental-jit=intepreter which sets the tier 2 interpreter to be used but not the jit

primal cypress
#

ah i see, I found this too

image.png
quartz orbit
#

How many people use AI to complete programming for their related work tasks

faint river
quartz orbit
spark magnet
fallen slateBOT
quartz orbit
#

Thank you for that then

frigid bison
#

In 3.10 the END_ASYNC_FOR opcode pops 7 values from the stack. Why is this 7 values? I understand that when an exception is triggered this is put on the stack, which in 3.10 is 3 values. So then we also have the iterator on the stack, so we should pop 4 values. After a lot of reading it seems like there are 2 exceptions on the stack somehow, can anyone clarify this?

simple sand
#

Should we refer to classes that conform to the __str__, __int__, etc. protocols as strables and intables respectably?

I'm thinking because at the current point the signature of print is print(*objects, sep=' ', end='\n', file=None, flush=False), but it's not clear what objects can actually be printed.

If we can change it to print(*strable, sep=' ', end='\n', file=None, flush=False) would it be clearer?

grave jolt
#

unless you go out of your way to define __str__ such that it raises an exception, which is arguably just a bad idea

clear hill
#

i would write that the object is convertible to str or int

simple sand
clear hill
#

I don’t see how that’s related to what I said

simple sand
clear hill
#

I could see cases where __str__ raising makes sense

#

__repr__ raising is more rude :p

simple sand
simple sand
gilded flare
grave jolt
#

yeah, let's not put even more stuff in the builtins namespace

#

we already have enough stinkers, like id

gilded flare
#

there's things in the stdlib other than the typing module that can be argued to have more use in builtins too

merry venture
#

well, search_function to be precise

#

hmm, i'll generally look around and see what is and isn't thread-safe in the encodings module

#

cc @uneven raptor

#

if the search function isn't thread-safe then .encode() might not be thread-safe as well...?

#

hmmm
although no
there would have to be different mods in the call of search_function for it to be a threat

#

that's a non-issue, nvm

clear hill
#

like, what is the nature of the thread-unsafety?

#

not clear just from the docs

crisp locust
boreal umbra
raven ridge
#

!warn 280050413833682944 The last two interactions you've had in this channel have both been rude (first reacting to someone's question about AI usage in the workplace with a vomiting emoji, and now telling someone to Google something). You should re-read our #code-of-conduct if you plan on continuing to engage with this server.

fallen slateBOT
#

:incoming_envelope: :ok_hand: applied warning to @crisp locust.

clear hill
#

in NumPy those HAVE_ variables are set at build time based on configure checks that meson does. I guess I could rename them to be e.g. NPY_HAVE_ to avoid problems like this too.

uneven raptor
#

mypyc defines Py_BUILD_CORE after pyport.h is included, so that would break in headers that assumed _Py_thread_local was available (by removing HAVE_THREAD_LOCAL checks)

#

I guess we could remove HAVE_THREAD_LOCAL entirely and that would fix it?

clear hill
#

well, at least numpy relied on that macro being unset, no idea if anything else depends on it

uneven raptor
#

could you create an issue? I guess the best fix is to only define that when Py_BUILD_CORE is set (but define _Py_thread_local unconditionally)

clear hill
clear hill
#

in NumPy we're dropping Python 3.11 support. Among other things, this lets us use the skip_file_prefixes keyword argument in warning.warns. I wrote a comment explaining why I think it's a good idea to wholesale switch over: https://github.com/numpy/numpy/issues/30311#issuecomment-3602664034. Anyone happen to be aware of cases where you'd ever want to use stacklevel instead of skip_file_prefixes?

spark verge
#

unraisable warnings

#

callbacks? generator based coroutines maybe

clear hill
#

what's an unraisable warning?

spark verge
#

Like when a file gets deleted without using .close()

#

The warning gets spat on the floor and pretty arbitrarily signed a module

clear hill
#

so a warning generated by a GC pass, I guess that's similar to a callback or a coroutine in that the runtime is calling it

#

NumPy has some C warning code that's a little gnarly but I don't think any of the Python implementation has any patterns like that.

spark verge
#

GC doesn't insert a frame so it's whatever is on the stack when the GC runs

hybrid relic
#

Does PCbuild have LTO on by default (Or does it even have LTO support at all)?

#

build.bat -h says nothing about LTO

crimson hatch
#

MSVC supports LTO, I believe PGO and LTO are done in release builds

hybrid relic
#

Ah it's an LTO is on when PGO is on and off otherwise sorta deal?

#

(Guessing because there is no LTO flag, only PGO)

silk wren
#

Salam

#

Hello

plush raptor
silk wren
plush raptor
silk wren
#

İ from azerbaycan

plush raptor
silk wren
silk wren
plush raptor
silk wren
#

I am dont racist

plush raptor
silk wren
#

I love this server, please introduce yourself.

plush raptor
astral gazelle
plush raptor
#

ok

feral pagoda
#

guys what do you think of my os

fallen slateBOT
# feral pagoda

Please react with ✅ to upload your file(s) to our paste bin, which is more accessible for some users.

feral pagoda
#

well its not rlly an os

#

but u get it

faint river
feral pagoda
#

mb i thought it was the career discussion thing

indigo canopy
#

I joined server 2 days ago why am I muted in group call channels

zenith topaz
mint cove
#

Are paramspecs supported in __call__? I'm getting type errors when attempting to migrate a class to Python 3.12 (PEP 695) syntax

#

It works fine with an explicit ParamSpec, but says expected "def (*Never, **Never) -> Never" with the 3.12 syntax...