#virtual environment created from 3.11rc1 in-tree build crashes on startup
fickle drift
~/programming/testing/docker-shenanigans/3.10
❯ sudo docker run --rm -it virtualenv-test-3.10:v1 cat /Python-3.10.6/venv/pyvenv.cfg
home = /usr/local
implementation = CPython
version_info = 3.10.6.final.0
virtualenv = 20.16.3
include-system-site-packages = false
base-prefix = /usr/local
base-exec-prefix = /usr/local
base-executable = /Python-3.10.6/python
~/programming/testing/docker-shenanigans/3.10 took 1s976ms
❯ sudo docker run --rm -it virtualenv-test-3.11:v4 cat /Python-3.11.0rc1/venv/pyvenv.cfg
home = /usr/local
implementation = CPython
version_info = 3.11.0.candidate.1
virtualenv = 20.16.3
include-system-site-packages = false
base-prefix = /usr/local
base-exec-prefix = /usr/local
base-executable = /Python-3.11.0rc1/python
not sure if this is relevant but here's the pyvenv.cfg files for both images
this looks increasingly like it's cpython's fault since virtualenv seems to be configuring it the same way...? I dunno how virtualenv works though
pure silo
you should compare with a file in regular venv
fickle drift
good idea
pure silo
The behavior changed when getpath was ported from C to Python:
https://github.com/python/cpython/commit/99fcf1505218464c489d419d4500f126b6d6dc28
This doesn't necessarily mean that the logic in getpath is at fault, it might be that Python is now leveraging some knowledge from python executable symlink/pyvenv.cfg that it didn't before which caused in-tree builds to stop working and the actual problem is in-tree build working incorrectly (i.e. setting invalid prefixes)
Like, I feel these prefixes really shouldn't be /usr/local when running Python in-place.
Once CPython is done building you will then have a working build that can be run in-place; ./python on most machines (and what is used in all examples), ./python.exe wherever a case-insensitive filesystem is used (e.g. on OS X by default), in order to avoid conflicts with the Python directory. There is normally no need to install your built copy of Python! The interpreter will realize where it is being run from and thus use the files found in the working copy.
This is excerpt from the Python's dev guide
fickle drift
Yeah, the prefix is suspicious and in fairness to getpath, prefix is actually a build-time variable
pure silo
To me it seems it doesn't realize where it is being run from and what's more important, it actually did not in 3.10 either
I don't know about earlier than that
fickle drift
I can try earlier versions given I now have a dockerfile template
pure silo
oh yeah, what I should also mention is that Python does not consider this as an in-tree build which can be seen when you start the interpreter in the created virtualenv
❯ python Could not find platform independent libraries <prefix> Could not find platform dependent libraries <exec_prefix> Python path configuration: PYTHONHOME = (not set) PYTHONPATH = (not set) program name = 'python' isolated = 0 environment = 1 user site = 1 safe_path = 0 import site = 1 is in build tree = 0 stdlib dir = '/usr/local/lib/python3.11' sys._base_executable = '/home/ichard26/Downloads/Python-3.11.0rc1/debug/python' sys.base_prefix = '/usr/local' sys.base_exec_prefix = '/usr/local' sys.platlibdir = 'lib' sys.executable = '/home/ichard26/programming/testing/black/3.11venv/bin/python' sys.prefix = '/usr/local' sys.exec_prefix = '/usr/local' sys.path = [ '/usr/local/lib/python311.zip', '/usr/local/lib/python3.11', '/usr/local/lib/python3.11/lib-dynload', ] Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding Python runtime state: core initialized ModuleNotFoundError: No module named 'encodings' Current thread 0x00007f241654f4c0 (most recent call first): <no Python frame>
fickle drift
❯ sudo docker run --rm -it virtualenv-test-3.11-stdlib-venv:v1 cat /Python-3.11.0rc1/venv/pyvenv.cfg
home = /Python-3.11.0rc1
include-system-site-packages = false
version = 3.11.0
executable = /Python-3.11.0rc1/python
command = /Python-3.11.0rc1/python -m venv /Python-3.11.0rc1/venv
huh
base-prefix is straight up not set
(also yes I know the docker image name is stupid, I just realized it's being prefixed with virtualenv still lol)
ah lol I forgot the ubuntu docker images don't come with an editor >.<
fickle drift
I'm out of ideas and this is also way outside my wheelhouse, so unless someone else who knows their way here wants to help out, I don't think I'll be able to do anything useful.
pure silo
I really thin there's enough info to go off from and I'm just hoping to interest Bernát enough to get a right issue to CPython
but if not then we can at least report the wrong value of in-tree build thing
fickle drift
https://github.com/ichard26/virtualenv-3.11rc1-in-tree-build-startup-crash dockerfiles I'm using just in case it's useful
fickle drift
I don't know about earlier than that
❯ sudo docker run --rm -it virtualenv-test-3.9:v1 cat /Python-3.9.13/venv/pyvenv.cfg
home = /usr/local
implementation = CPython
version_info = 3.9.13.final.0
virtualenv = 20.16.3
include-system-site-packages = false
base-prefix = /usr/local
base-exec-prefix = /usr/local
base-executable = /Python-3.9.13/python
~/programming/testing/docker-shenanigans/3.9 on main [?] took 1s844ms
❯ sudo docker run --rm -it virtualenv-test-3.9:v1 /Python-3.9.13/venv/bin/python -c "import sys; print(sys.prefix, sys.base_prefix)"
/Python-3.9.13/venv /usr/local
3.9 doesn't seem to know either
fickle drift
I really thin there's enough info to go off from and I'm just hoping to interest...
Sorry for bugging you @tribal oyster, but do you think this is worth debugging / investigating more? I don't know whether creating a virtual environment from an in-tree build of CPython is a supported use-case, but it was working before so it is particular.
If it's not a supported use-case, then that's simple enough 🙂
tribal oyster