PYTHON / MODULES AND PACKAGES
Packages, __init__.py, and namespaces
Build real multi-file packages, use __init__.py to define a package's public surface, and tell regular packages from PEP 420 namespace packages.
What you will learn
- See a package as a module whose body is __init__.py and whose search list is __path__
- Re-export chosen names in __init__.py so callers write `from pkg import thing`
- Import submodules explicitly with `import pkg.sub` or `from pkg import sub`
- Identify a namespace package from its _NamespacePath and missing __file__
Understanding Packages, __init__.py, and namespaces
A package is not a special kind of object: it is an ordinary module that happens to carry a `__path__` attribute. When you import `shapes.circle`, the import system first imports `shapes` by scanning `sys.path`, then looks for `circle` only inside the directories listed in `shapes.__path__`. Every extra dot narrows the search to the parent's `__path__`, which is why nested submodules are never searched for on `sys.path` again, and why two unrelated projects can both ship a `utils.py` without colliding.
For a regular package, the directory contains `__init__.py`, and that file is the module body: its top-level code runs once, the first time anything under the package is imported, and the resulting module object is cached in `sys.modules` under the name `shapes`. That makes `__init__.py` the natural place to pull a few names up from submodules (`from .circle import area`) so users of your package do not need to know your internal file layout. Keep it cheap: every import of any submodule pays for the parent's `__init__.py` first, so heavy work or imports of every submodule there slows down the whole package and invites circular imports.
Since PEP 420, a directory with no `__init__.py` can still be imported, but it behaves differently. Such a directory is treated as a *portion*: the finder records it and keeps scanning the rest of `sys.path`, so several directories with the same name on different roots merge into one package whose `__path__` is a dynamic `_NamespacePath` with multiple entries and whose `__file__` is absent. That is exactly what you want when separately installed distributions must share one prefix such as `acme.auth` and `acme.billing`, and exactly what you do not want by accident, because a stray same-named folder can silently join your package.
import sys, tempfile
from pathlib import Path
root = Path(tempfile.mkdtemp())
pkg = root / "shapes"
pkg.mkdir()
pkg.joinpath("__init__.py").write_text(
"print('-> shapes/__init__.py runs')\n"
"from .circle import area\n"
"__all__ = ['area']\n"
)
pkg.joinpath("circle.py").write_text(
"print('-> shapes/circle.py runs')\n"
"PI = 3.14159\n"
"def area(r):\n"
" return PI * r * r\n"
)
sys.path.insert(0, str(root))
import shapes
print('area(2) =', round(shapes.area(2), 4))
print('__package__ =', shapes.__package__)
print('__path__ points at =', Path(list(shapes.__path__)[0]).name)
print('__file__ is =', Path(shapes.__file__).name)
print('shapes.circle cached:', 'shapes.circle' in sys.modules)
print('shapes.circle is', shapes.circle.__name__)A package is a module with a `__path__`: `__init__.py` is its body, and `__path__` is the list of directories searched for its submodules.
Worked examples
import pkg does not import pkg.sub
Shows that a submodule is only reachable after something actually imports it.
import sys, tempfile
from pathlib import Path
root = Path(tempfile.mkdtemp())
pkg = root / "toolkit"
pkg.mkdir()
pkg.joinpath("__init__.py").write_text("VERSION = '1.0'\n")
pkg.joinpath("text.py").write_text("def shout(s):\n return s.upper() + '!'\n")
sys.path.insert(0, str(root))
import toolkit
print(toolkit.VERSION)
try:
print(toolkit.text.shout('hi'))
except AttributeError as e:
print('AttributeError:', e)
import toolkit.text
print(toolkit.text.shout('hi'))Example explained
Line 1`import toolkit` only executes `__init__.py`, which defines VERSION and nothing else.
Line 2`toolkit.text` fails because no code has bound `text` as an attribute of the package yet.
Line 3`import toolkit.text` loads the submodule and, as a side effect, sets it as an attribute on `toolkit`.
Line 4Adding `from . import text` to `__init__.py` would make the first attempt work.
Two directories, one namespace package
Demonstrates PEP 420 portions merging into a single package with a dynamic __path__.
import sys, tempfile
from pathlib import Path
a, b = Path(tempfile.mkdtemp()), Path(tempfile.mkdtemp())
(a / "nsdemo").mkdir()
(a / "nsdemo" / "alpha.py").write_text("name = 'alpha'\n")
(b / "nsdemo").mkdir()
(b / "nsdemo" / "beta.py").write_text("name = 'beta'\n")
# note: neither nsdemo directory contains an __init__.py
sys.path[:0] = [str(a), str(b)]
import nsdemo
import nsdemo.alpha
import nsdemo.beta
print('type of __path__:', type(nsdemo.__path__).__name__)
print('__file__:', getattr(nsdemo, '__file__', None))
print('portions:', len(list(nsdemo.__path__)))
print(nsdemo.alpha.name, nsdemo.beta.name)Example explained
Line 1Without `__init__.py`, each `nsdemo` directory is recorded as a portion instead of ending the search.
Line 2`__path__` is a `_NamespacePath`, not a plain list, and it re-checks `sys.path` when that changes.
Line 3There is no `__file__` because no single file is the package's body.
Line 4`alpha` and `beta` live on different roots yet resolve under one dotted name.
Relative imports between siblings
Shows why a submodule must say `from . import sibling` rather than `import sibling`.
import sys, tempfile
from pathlib import Path
root = Path(tempfile.mkdtemp())
pkg = root / "app"
pkg.mkdir()
pkg.joinpath("__init__.py").write_text("")
pkg.joinpath("config.py").write_text("LIMIT = 5\n")
pkg.joinpath("worker.py").write_text(
"try:\n"
" import config\n"
"except ModuleNotFoundError as e:\n"
" print('absolute failed:', e)\n"
"from . import config\n"
"print('relative worked, LIMIT =', config.LIMIT)\n"
"print('__package__ =', __package__)\n"
)
sys.path.insert(0, str(root))
import app.workerExample explained
Line 1`import config` searches `sys.path` only, and `app/` itself is not on `sys.path`, so it fails.
Line 2`from . import config` uses `__package__` ('app') to build the real name `app.config`.
Line 3This is why the package directory should never be added to `sys.path` as a workaround: it would create two copies of the same module.
Important notes
A regular package's `__path__` is an ordinary list fixed when `__init__.py` runs, while a namespace package's `_NamespacePath` recomputes itself when `sys.path` changes, so directories added later can appear.
A namespace package cannot contain `__init__.py`: as soon as one candidate directory has it, that directory becomes the whole package and any portions collected from other roots are discarded.
Common mistakes
Calling `pkg.sub.fn()` after only `import pkg`: unless `__init__.py` imported `sub`, you get `AttributeError: module 'pkg' has no attribute 'sub'`.
Writing `import config` inside a package module to reach a sibling: Python 3 has no implicit relative imports, so it raises ModuleNotFoundError, and 'fixing' it by adding the package folder to sys.path loads the same file twice under two names.
Forgetting `__init__.py` while another directory with the same name sits elsewhere on sys.path: the two silently merge into one namespace package and submodules resolve to files you never intended to ship.
Try it yourself
Change, predict, then run
Using the `write_text` pattern from the examples, build a package `geometry` with `rect.py` and `circle.py`, each defining an `area` function, and make `import geometry` alone expose both `geometry.rect_area` and `geometry.circle_area`; then print `sorted(m for m in sys.modules if m.startswith('geometry'))`.
Open the Python workspaceCheck your understanding
Two directories on `sys.path` both contain a folder named `plugins`. The one appearing later on `sys.path` has an `__init__.py`; the earlier one does not. What does `import plugins` give you?
- The regular package with `__init__.py`, and the other directory is ignored entirely
- The earlier directory wins because sys.path order decides, so you get a namespace package
- A namespace package whose `__path__` contains both directories
- An ImportError, because two directories on sys.path claim the same package name
Show answer
A directory without `__init__.py` is only recorded as a namespace *portion*; the finder keeps scanning. When it later finds `plugins/__init__.py`, that is a real spec, so the search stops and all accumulated portions are thrown away. The 'first on sys.path wins' rule that holds for plain modules does not apply here, which makes option 2 tempting but wrong.