#Python 3.13t and tox

1 messages · Page 1 of 1 (latest)

somber coral
#

That seems likely, but how do I fix it? Tox is making the wheel, no? Using the 3.13t interpreter? I think there's a lot about tox machinery I don't understand.

calm dagger
#

maybe tox needs updating for free-threading?

somber coral
#

not seeing any issues written, but I can start there.

fading dagger
#

Shouldnt that just trigger reenabling the gil? I thought i read a section on importing modules not ready for freethreading that described that happening

#

(If it was just that its not a.3.13t wheel)

somber coral
#

I don't understand "reenabling the gil"? I want to be sure to test with a version with no gil.

fading dagger
#

Still need a 3.13t wheel.for your needs,.but lack of one shouldnt do this

somber coral
#

tbh, i would rather that it do this. I don't like falling back in hard-to-see ways.

fading dagger
#

See the. environment variable immediately below for that

#

Ill be back at my desk in a few, gonna poke at this

somber coral
fading dagger
#

Should have said below that in pep, on mobile for a few and was still sortof in.abberviated text thought mode

somber coral
#

hmm, now I'm wondering about adding Py_mod_gil to my C extension.

calm dagger
fading dagger
#

Just got back to my desk, I see that was actually tox output not interpreter output, but that should still be installable for free threaded python, just would disable the free-threading.

calm dagger
fading dagger
#

probably, I just got done reviewing another module getting finished for 3.13 last night (audioop-lts)

#

but was on mobile and wasn't remembering where all the stuff I referenced was :/

fading dagger
#

I'll come back to this after lunch, there's something going on here that doesn't seem right, but it's not really just one thing. I'd expect setuptools to have built a 3.13t wheel, and I'd also have seperately expected pip to not reject a wheel for differing on 313/3.13t

it doesn't currently appear to be a fault of tox, but there could be something tox has done with how it handles environments and I havent gotten that far yet.

somber coral
hoary juniper
#

My guess is that it's not pip, nor tox, but it's the build backend itself generating the wrong wheel tag?

#

@somber coral What is the wheel tag if you do a python -m build on a 3.13t virtual env?

#

Bah, joys of being on mobile is random capitalizations.

somber coral
hoary juniper
#

Hmm... And editable wheel?

#

Because this looks like tox is asking for an editable wheel.

#

Bah, build doesn't expose that in a CLI.

#

Does a pip install -e . work?

somber coral
# 
% pip install -e .
Obtaining file:///Users/ned/coverage/trunk
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Building wheels for collected packages: coverage
  Building editable for coverage (pyproject.toml) ... done
  Created wheel for coverage: filename=coverage-7.6.1a0.dev1-0.editable-cp313-cp313t-macosx_14_0_arm64.whl size=9437 sha256=0cd67dc40d39e2c39c11836f4a4c0213ab20980d526c01fde6120304d8a605a6
  Stored in directory: /private/var/folders/6j/khn0mcrj35d1k3yylpl8zl080000gn/T/pip-ephem-wheel-cache-fo6c8crt/wheels/3a/6d/ea/66d4b328dc1a5be5c68cf7d8639e26777023337692e0d3579a
Successfully built coverage
Installing collected packages: coverage
Successfully installed coverage-7.6.1a0.dev1
hoary juniper
#

Huh. Thanks for the quick responses. I think this is tox, I wrote a comment over on the tox issue as well to reflect some of that (I might rephrase that/follow up there when I've got slightly less split attention).

somber coral
hoary juniper
#

Yea, I'm realising that and not particularly pleased with how this is playing out in that regard. 🙈

somber coral
hoary juniper
#

Ok, in front of a computer now. Taking a closer look...

somber coral
hoary juniper
#

(BTW, this anypy trick is quite neat!)

#

Recreated a clean env with a new tox and a purged pip cache to get the same "it works" result. :/

somber coral
#

hmm, that is too bad... i wonder what the difference is. How did you build 3.13t?

#

maybe this is a pyenv problem?

hoary juniper
#

asdf install python 3.13.0rc1t -- this also uses python-build that pyenv uses.

#

(rebuilding that to have logs for my build...)

# 
❯ asdf install python 3.13.0rc1t  
python-build 3.13.0rc1t /Users/pradyunsg/.asdf/installs/python/3.13.0rc1t
python-build: use openssl@3 from homebrew
python-build: use readline from homebrew
Downloading Python-3.13.0rc1.tar.xz...
-> https://www.python.org/ftp/python/3.13.0/Python-3.13.0rc1.tar.xz
Installing Python-3.13.0rc1...
python-build: use readline from homebrew
python-build: use zlib from xcode sdk
Installed Python-3.13.0rc1 to /Users/pradyunsg/.asdf/installs/python/3.13.0rc1t

Installing default python packages...
[notice] A new release of pip is available: 24.1.1 -> 24.2
[notice] To update, run: pip install --upgrade pip
asdf: Warn: You have configured asdf to preserve downloaded files (with always_keep_download=yes or --keep-download). But
asdf: Warn: the current plugin (python) does not support that. Downloaded files will not be preserved.

Bah, I did not file the PR to update pip in ensurepip. 🤦🏻‍♂️

#

Nothing useful in the logs, I guess?

# 
-[checkout]/.tox/.pkg-cpython313/lib/python3.13/site-packages/setuptools/command/editable_wheel.py:357: InformationOnly: Editable installation.
+[checkout]/.tox/.pkg-cpython313/lib/python3.13t/site-packages/setuptools/command/editable_wheel.py:357: InformationOnly: Editable installation.

Huh, your output has it invoking setuptools on 3.13, not 3.13t. That'd explain the 3.13 wheel.

somber coral
hoary juniper
#

Not sure, trying to look through the diff of our logs to take an educated guess. 😅

# 
-.pkg-cpython313: 6366 D accepted PathPythonInfo(spec=CPython3.13.0.candidate.1-64, exe=/usr/local/bin/python3.13, platform=darwin, version='3.13.0rc1 (main, Aug  3 2024, 08:38:30) [Clang 15.0.0 (clang-1500.3.9.4)]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:81]
+.pkg-cpython313: 7023 D accepted PathPythonInfo(spec=CPython3.13.0.candidate.1-64, system=[base-prefix]/bin/python3, exe=/Users/[user]/.asdf/shims/python3, platform=darwin, version='3.13.0rc1 experimental free-threading build (main, Aug  3 2024, 20:15:48) [Clang 15.0.0 (clang-1500.3.9.4)]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:81]

Tox resolved to the 3.13 on your path, rather than 3.13t, which makes sense.

#

OH, maybe a wild idea: try running with the pyenv shim and only python3.13.0rc1t on the PATH (i.e. something like pyenv shell 3.13.0rc1t if I remember the invocation correctly)?

#

Maybe also 3.8 for tox itsefl.

somber coral
#

hmm, that might work, but will be a big pain. Why is tox finding python3.13?

#

when i move python3.13 aside I get, anypy: skipped because could not find python interpreter with spec(s): cpython313 for package environment .pkg

#

what did you mean by "which makes sense"?

hoary juniper
#

Tox resolved to the 3.13 on your path, rather than 3.13t, which makes sense.
Oh, that it explains the behaviour we're seeing, not that the behaviour is correct. 😅

somber coral
#

ok, got it

hoary juniper
# 
-.pkg-cpython313: [time] D discover PATH[0]=/usr/local/virtualenvs/coverage/bin [virtualenv/discovery/builtin.py:148]
-.pkg-cpython313: [time] D discover PATH[1]=/Users/[user]/bin [virtualenv/discovery/builtin.py:148]
-.pkg-cpython313: [time] D discover PATH[2]=/Users/[user]/.local/bin [virtualenv/discovery/builtin.py:148]
-.pkg-cpython313: [time] D discover PATH[3]=/usr/local/bin [virtualenv/discovery/builtin.py:148]
+.pkg-cpython313: [time] D discover PATH[0]=/Users/[user]/.asdf/shims [virtualenv/discovery/builtin.py:148]

Your tox runs' search on PATH ends up finding a bunch of locations to look for the interpreter at vs mine, and I'm gonna guess that /usr/local/bin/python3.13 is a global install of Python 3.13?

#

That's what it seems to end up picking up at the end... and the behaviour is different after this point AFAICT.

somber coral
hoary juniper
#

Another wild idea for testing theory: if you make that symlink point to 3.13t for a few minutes, does the failure resolve?

somber coral
hoary juniper
# calm dagger maybe tox needs updating for free-threading?

Ok, then I do think that Hugo got it right! Tox needs to be aware of using the right interpreter type (nogil vs regular) when looking for one to create the .pkg-cpython313 virtualenv here (and maybe also stick the t in the created venv's name).

/cc @pallid badger for awareness since I dunno how much of this is worth reproducing over the issue tracker. 😅

somber coral
#

thanks for the help!

calm dagger
#

fwiw I can also repro the error using the official macOS installer, and running COVERAGE_ANYPY=/usr/local/bin/python3.13t tox -re anypy

hoary juniper
#

@calm dagger OK to quote you on the issue?

calm dagger
hoary juniper
#

thank_you -- posted on the tox issue.

somber coral
#

Is there an official place to find "3.13t" as a string in sys or platform or sysconfig etc?

hoary juniper
somber coral
#

ok, but "abiflag" is a good word to know, and sys.abiflags is helpful.

hoary juniper
#

I think packaging looks at Py_GIL_DISABLED which is what the PEP set as the marker boolean?

somber coral
#

oh, but only unix

hoary juniper
#

Oh, abiflags? Yea.

somber coral
#

that's actually ok for me, this is helpful.

hoary juniper