🔒 Static Python Compilation? | Python | Page 1

abstract trellis Dec 9, 2024, 1:10 AM

#

I have an app using nuitka, and for fun and knowledge I want to attempt to port it off nuitka and onto a native python installation that includes everything necessary.

I gather the means to do this directly would be to start with a static build of python.

I started by using this:


cython -3 hello.py --embed

# cd $HOME/GitHub/cpython

# PY_INCLUDES="$(python -m python-config --includes)"
# PY_CFLAGS="$(python -m python-config --cflags)"
# PY_LFLAGS="$(python -m python-config --ldflags)"

# cd -

PY_INCLUDES="$(python3.12-config --includes)"
PY_CFLAGS="$(python3.12-config --cflags)"
PY_LFLAGS="$(python3.12-config --ldflags --embed)"
PY_PATH="-L/usr/lib/python3.12/config-3.12-x86_64-linux-gnu/"

echo Compiling python with cflags: $PY_CFLAGS lflags: $PY_LFLAGS from path $PY_PATH

gcc --static $PY_INCLUDES $PY_CFLAGS hello.c -o hello \
    -static-libgcc -static-libstdc++ $PY_PATH $PY_LFLAGS

To dump a simple hello.py script into an excutable. It worked until native modules started loading:

Traceback (most recent call last):
  File "hello.py", line 1, in init hello
    import base64
  File "/usr/lib/python3.12/base64.py", line 10, in <module>
    import struct
  File "/usr/lib/python3.12/struct.py", line 13, in <module>
    from _struct import *
ImportError: /usr/lib/python3.12/lib-dynload/_struct.cpython-312-x86_64-linux-gnu.so: undefined symbol: PyObject_ClearWeakRefs

I'm not exactly sure, but this reads like it's still trying to load the system modules here. So, I grabbed a static build of 3.10 out of the AUR (I, uh, couldn't get it building statically myself after digging up a few pages on it) and get this:

/usr/bin/ld: /tmp//ccAdPPcj.ltrans0.ltrans.o: in function `__pyx_pymod_exec_hello':
/home/sk3shun-8/GitHub/umomwd/hello.c:4168:(.text.unlikely+0x9ba): undefined reference to `Py_Version'
collect2: error: ld returned 1 exit status

Any suggestions greatly appreciated 🙂

white cometBOT Dec 9, 2024, 1:10 AM

#

abstract trellis I have an app using nuitka, and for fun and knowledge I want to attempt to port ...

#

@abstract trellis

Python help channel opened

Remember to:

Ask your Python question, not if you can ask or if there's an expert who can help.
Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

green jasper Dec 9, 2024, 1:44 AM

#

abstract trellis I have an app using nuitka, and for fun and knowledge I want to attempt to port ...

Not sure it would help It looks like it's getting trouble at linking and finding symbols. Have you tried ensuring the missing symbols are in the binaries with a tool like nm ?

abstract trellis Dec 9, 2024, 2:09 AM

#

Huh, yeah, my executable itself does have pyObject_ClearWeakRefs:

000000000053c360 T PyObject_ClearWeakRefs
0000000000447dce t PyObject_ClearWeakRefs.cold

Or, it looks like it does. I was under the impression -rdynamic should solve that, perhaps I placed it wrong?

#

The main reason I thought I needed my own static build was the extra modules all being dynamics still:
Seems rather pointless that my distro provides a static build of python itself if this is how it's behaving...

#

It still says that the static lib is doing a bunch of dlopen calls during comptime... so it makes sense it's failing this way:

/usr/bin/ld: /usr/lib/python3.12/config-3.12-x86_64-linux-gnu//libpython3.12.a(dynload_shlib.o): in function `_PyImport_FindSharedFuncptr':
(.text+0xb6): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: /usr/lib/python3.12/config-3.12-x86_64-linux-gnu//libpython3.12.a(posixmodule.o): in function `os_getgrouplist':
(.text.unlikely+0xdd64): warning: Using 'getgrouplist' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: /usr/lib/python3.12/config-3.12-x86_64-linux-gnu//libpython3.12.a(posixmodule.o): in function `os_initgroups':

It seems like it's not built right for this, to me.

abstract trellis Dec 9, 2024, 2:40 AM

#

After carving away some of the compile flags suggested by python-config I get this:

Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Python path configuration:
  PYTHONHOME = (not set)
  PYTHONPATH = (not set)
  program name = './hello'
  isolated = 0
  environment = 1
  user site = 1
  import site = 1
  sys._base_executable = '/home/sk3shun-8/GitHub/umomwd/hello'
  sys.base_prefix = '/usr'
  sys.base_exec_prefix = '/usr'
  sys.platlibdir = 'lib'
  sys.executable = '/home/sk3shun-8/GitHub/umomwd/hello'
  sys.prefix = '/usr'
  sys.exec_prefix = '/usr'
  sys.path = [
    '/usr/lib/python310.zip',
    '/usr/lib/python3.10',
    '/usr/lib/lib-dynload',
  ]

Which, ah. I guess is an improvement. Sort of.

white cometBOT Dec 9, 2024, 3:40 AM

#

Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.

#🔒 Static Python Compilation?