🔒 How to turn a modules into a shared libs for monorepo

[manic hemlock]

I'm currently trying to change directory of my python modules called x_service.py and y_service.py from the python project directory, into adjacent folder that supposed to be the shared library location called libs/ so that those modules can be reusable if by any chance needed by other project.

this is my planned directory so that x_service and y_service can be reusable which was moved previously from python-proj/ into libs/.

.
└── big-project/
    ├── some service
    ├── some service
    ├── libs/
    │   └── python/
    │       ├── x-service/
    │       │   ├── __init__.py
    │       │   └── x_service.py
    │       ├── y-service/
    │       │   ├── __init__.py
    │       │   └── y_service.py
    │       └── setup.py
    └── python-proj/
        └── src/
            ├── config/
            │   └── container.py
            └── main.py

But somehow I keep getting Import error in VS Code when trying to do this, did i miss some step or knowledge? or it's basically can't be done?

from libs.python.x_service.x_service import X_Service_Class
from libs.python.y_service.y_service import Y_Service_Class

[dim heath]

@manic hemlock

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[sand ibex]

you can only import from libs if the root project is big-project

i.e. big-project/__init__.py exists

and you invoke python -m big_project.something

you can also use pip to locally install these modules

or you can add that directory to your sys.path

[manic hemlock]

oh i see, yeah sadly big-project is the project repo that contains several backend config and microservices, including the python-project

[manic hemlock]

but it doesn't mean i need to upload it to PyPI right?
I've tried to use that setup.py files before trying to pip install

from setuptools import setup, find_packages

setup(
    name="python-infrastructure",
    version="0.1.0",
    packages=find_packages(where='.'),
    install_requires=[
        "x_dependency~=2.0",  # Compatible with 2.x versions
        "y_admin~=6.6", # Compatible with 6.6.x versions
    ],
)

however, as far as the python-infrastructure package includes in my virtual environment, those modules from both x_service and y_service is not there and can't be imported. is it something wrong with how i write my setup.py file?

[sand ibex]

but it doesn't mean i need to upload it to PyPI right?
Correct, no upload (or publication) required.

setup.py is not recommended in modern python (I also don't have any experience with that, so I'm not sure if you have anything wrong there)

[manic hemlock]

that's interesting point. so how can i actually do pip install for those custom made modules if given the project tree above to follow?

[sand ibex]

Here's one solution:

big-project/
├── service1
├── service2
├── libs/
│   └── python/
│       └── x_service/
│           ├── src/
│           │   └── x_service/
│           │       ├── __init__.py
│           │       └── other_stuff.py
│           ├── tests/
│           │   └── optional_tests.py
│           └── pyproject.toml
└── python-project/
    └── src/
        └── main.py

your pyproject.toml would look something like this (minimal example)

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "x_service"

and then you can pip install -e big-project/libs/python/x_service

(you could use a venv and install it only within the python-project if you want)

you can also put -e big-project/libs/python/x_service in your requirements.txt (though you might have to use an abolute path)

the -e flag stands for "editable", so pip will only install a symlink to the original files, and you can edit them in-place without reinstalling

and yes x_service/src/x_service/* is redundant, that's just how it is
if both x and y services are "related" at all, you can just put both of those services in one module and install them both together (i.e. services.x and services.y) to reduce the amount of boilerplate

[manic hemlock]

it's a complete walkthrough, thankss a lot @sand ibex for the help! I'm implementing it right now. I'll update the result soon!

[sand ibex]

np, good luck

[dim heath]

Python help channel closed for inactivity

This help channel has been closed. Feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.