🔒 The purpose of Namespace Packages

[frozen ivy]

I am not sure what they are for, I'll write my question towards the end of this post.

I have read the folowing 2 links on namespace packages:

https://docs.python.org/3/reference/import.html#namespace-packages

https://peps.python.org/pep-0420/

So if I tried to summarize my mental model of regular packages and namespace packages it is this:

1️⃣ Starting from the top-most directory, down to all its nested subdirectories. Python creates 1 package to correspond with every directory.

2️⃣ For some reason currently unknown to me, the person who wrote the documentation insists that we don't treat packages as directories. I don't want to bother thinking too deeply about that. I guess they just meant to say that python looks at a project's directory structure when making a decision of how many packages to create and what to call them.

3️⃣ If a directory contains an __init__.py file, it is a regular package. Otherwise it is a namespace package.

4️⃣ All distinct directories corresponding to the same namespace package (directories that share the same name in any directory that can be reached from one of the paths inside sys.path) have all their code mixed together into a package that is created on the spot.

5️⃣ This enables developers to use packages/modules located in directories not nested anywhere inside the top-most directory of their python project.

My tentative conclusion (mental model):

If we don't supply an init file in a directory (corresponding to a package), then we are telling Python to load the associated package from directories we list on sys.path.

Namespace packages are sort of virtual version of a package that can be either fetched from a directory that doesn't need to be nested in the top-most directory of a project, fetched from LAN or internet, or extracted from a compressed file.

What would be the incentive to use namespace package? What if two different directories share the same API? Does 1 get priority?

[urban delta]

@frozen ivy

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[frozen ivy]

Revision: Rather than API, I guess the question makes more sense if I instead asked, can a Namespace Package have 2 Modules with the same name? Does Python rename any Modules with the same name or discard one of the two+ same name modules?

[stark iris]

What would be the incentive to use namespace package?
personally i rarely encounter them so i dont have a thorough understanding of their usage, but as i understand it, their main purpose is being able to organize modules that come from different sources into a single package, instead of having to rely only on top-level packages and follow some naming convention like superproject_abc / superproject_xyz

i think package management in linux distros is a good example since you can have both modules installed by the system (like via apt) and modules installed by the user (via pip) share a namespace package, even though they physically exist in different directories (system-wide dist-packages/ and user-specific site-packages/ respectively)

implicit namespace packages also makes it more compatible with package managers according to that pep, since they apparently don't allow files to conflict, breaking the old pkgutil method with __init__.py

[frozen ivy]

1. True, for example, I guess two different projects could share the same source code (in the filesystem) from a common package they both use.

2. Or a package manager can automatically download and compile any missing dependencies (packages it relies on) of a package a user wants to install.

[stark iris]

i would imagine whichever namespace is first in sys.path gets priority, just like normal sys.path behaviour

[frozen ivy]

Arigatou Hakase.

[stark iris]

looks like this is the case: dir1/ └── namespace/ └── mod.py dir2/ └── namespace/ └── mod.py ```sh

set "PYTHONPATH=dir1;dir2"
py -m namespace.mod
mod from dir1
set "PYTHONPATH=dir2;dir1"
py -m namespace.mod
mod from dir2```

[frozen ivy]

I just found this website. I'll probably read it thoroughly at my own pace later. I have a lot of learning to do on distributions:

https://packaging.python.org/en/latest/overview/

[stark iris]

ah yes, there are a bunch of options for making those

personally i stick to the setuptools build backend since they adopted the pyproject.toml standard, but poetry / hatch / pdm are alternatives too

[frozen ivy]

The irony is that I am more familiar with Containers, Linux Virtualization, and Orchestrators than I am with the primitive distribution package managers (though I guess managers are actually used within containers).

[stark iris]

ive recommended these links before for getting started with packaging projects
https://setuptools.pypa.io/en/latest/userguide/quickstart.html#basic-use
https://packaging.python.org/en/latest/guides/writing-pyproject-toml/
(PyPUG's Packaging Python Projects tutorial is more comprehensive than setuptools's quickstart, but i still link the quickstart since its short)

a bit odd that they dont cover installing from source just after getting pyproject.toml set up...

although i guess that only matters if you use src-layout over flat-layout (see also setuptools automatic discovery and PyPUG's comparison)

all of my newer projects start off with src-layout, something like: my_project/ ├── src/ │ └── hello_world/ │ ├── __init__.py │ └── __main__.py └── pyproject.toml where pyproject.toml contains at least: ```toml
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

[project]
name = "hello-world"
version = "1.0.0"and to get everything ready for further development:sh
/my_project $ python -m venv .venv
/my_project $ source .venv/bin/activate
(.venv) /my_project $ pip install --editable .
...
Installed hello-world-1.0.0

(.venv) /my_project $ python -m hello_world
Hello world from main.py!```

-e/--editable is documented on setuptools's site if you want to read that:
https://setuptools.pypa.io/en/latest/userguide/development_mode.html

[urban delta]

Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.