🔒 Modular App, Bundling and installing module dependencies.

[deft saddle]

I'm developing a modular Electron-based application with a JavaScript frontend/UI and a Python backend. Both components establish REST servers, effectively serving as makeshift inter-process communication (IPC).

Neither this core app, nor the modules have any knowledge of each other. The modules are installed by the user and integrate into the process via dynamic imports.

For deployment, the Python backend is compiled into a standalone executable (.exe) using PyInstaller. Just like the core, the modules have separate JavaScript and Python components. The Python part of each module is compiled into a .pyd file.

A key issue arises when importing these .pyd files, especially if the modules have Python dependencies that aren't included in the core backend. To manage this, each shipped module comes bundled with its own requirements.txt file listing these additional dependencies.

However, when attempting to programmatically install these dependencies at runtime using an internal instance of pip (packaged with the core backend executable via hiddeninstalls), I consistently encounter errors. This installation step currently fails, blocking seamless module integration.

I'm building the backend core using

    pyinstaller --onedir --noconfirm --name=paige_backend_server \
        --hidden-import=pip \
        --hidden-import=pip._internal \
        src/core/rest_server.py

although I've had as much as 25 more lines to try to get this to work.

In the app I then have pip._internal.main(["install", "--target", userDir, "-r", tmp.name]) (with tmp.name being the requriements.txt file)

This then leads to a "ModuleNotFoundError: No module named 'pip._internal.commands.install' error. This is changed by adding pip._internal.commands.install to the --hidden-import but again, even after adding 25 additional --hidden-imports, I get errors like'FileNotFoundError: [Errno 2] No such file or directory

[elfin magnet]

Hey @deft saddle!

Please edit your message to use a code block

Make sure there are no spaces between the back ticks and py. Make sure you put your code on a new line following py. There must not be any spaces after py.

Here is an example of how it should look:
```py
print('Hello, world!')
```

This will result in the following:

print('Hello, world!')```

[elfin magnet]

@deft saddle

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[elfin magnet]

@deft saddle

Python help channel closed for inactivity

This help channel has been closed. Feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.