Python Packaging

Proper Python packaging makes your code installable, testable, and distributable. This page covers the modern Python packaging standard: pyproject.toml, the src layout, and versioning best practices.

The Modern Python Project

code

my-package/
├── pyproject.toml          # Project metadata and configuration
├── README.md
├── LICENSE
├── src/
│   └── my_package/
│       ├── __init__.py     # Package init (exposes version)
│       ├── core.py         # Main module
│       ├── utils.py        # Utilities
│       └── py.typed        # PEP 561 marker (supports type checking)
├── tests/
│   ├── __init__.py
│   ├── test_core.py
│   └── conftest.py
└── .github/
    └── workflows/
        └── ci.yml

pyproject.toml

The pyproject.toml file replaces setup.py, setup.cfg, and requirements.txt with a single, standardized configuration file.

toml

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "my-package"
version = "0.1.0"
description = "A package for doing amazing things with AI"
readme = "README.md"
license = "MIT"
requires-python = ">=3.11"
authors = [
    { name = "Your Name", email = "you@example.com" },
]
keywords = ["ai", "llm", "tools"]
classifiers = [
    "Development Status :: 3 - Alpha",
    "Intended Audience :: Developers",
    "License :: OSI Approved :: MIT License",
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
]

dependencies = [
    "httpx>=0.27",
    "pydantic>=2.0",
    "rich>=13.0",
]

[project.optional-dependencies]
dev = [
    "pytest>=8.0",
    "pytest-asyncio>=0.23",
    "ruff>=0.4",
    "mypy>=1.10",
]
ml = [
    "torch>=2.2",
    "transformers>=4.40",
    "datasets>=2.19",
]

[project.scripts]
my-cli = "my_package.cli:main"

[project.urls]
Homepage = "https://github.com/you/my-package"
Documentation = "https://my-package.readthedocs.io"
Repository = "https://github.com/you/my-package"
Changelog = "https://github.com/you/my-package/blob/main/CHANGELOG.md"

[tool.hatch.build.targets.wheel]
packages = ["src/my_package"]

[tool.ruff]
line-length = 100
target-version = "py311"

[tool.ruff.lint]
select = ["E", "F", "I", "N", "UP", "B", "SIM"]

[tool.pytest.ini_options]
testpaths = ["tests"]
asyncio_mode = "auto"

[tool.mypy]
python_version = "3.11"
strict = true

The src Layout

The src/ layout prevents a critical issue: accidentally importing your package from the source tree instead of the installed version.

python

# ❌ Flat layout (problems during development)
my-package/
├── my_package/
│   ├── __init__.py
│   └── core.py
├── tests/
│   └── test_core.py
# Running `python -c "import my_package"` imports from ./my_package/
# NOT from the installed package — bugs may go undetected!

# ✅ src layout (forces installed-package testing)
my-package/
├── src/
│   └── my_package/
│       ├── __init__.py
│       └── core.py
├── tests/
│   └── test_core.py
# Must install the package first: pip install -e .
# Tests always use the installed version

Package Init and Versioning

python

# src/my_package/__init__.py
"""My Package - A package for doing amazing things with AI."""

from importlib.metadata import version, PackageNotFoundError

try:
    __version__ = version("my-package")
except PackageNotFoundError:
    # Package is not installed
    __version__ = "0.0.0"

from my_package.core import AmazingClient, process_data

__all__ = ["AmazingClient", "process_data", "__version__"]

Dynamic Versioning with Hatchling

toml

# pyproject.toml
[project]
name = "my-package"
dynamic = ["version"]

[tool.hatch.version]
path = "src/my_package/__init__.py"

# Or use git tags for versioning
[tool.hatch.version]
source = "vcs"

[tool.hatch.build.hooks.vcs]
version-file = "src/my_package/_version.py"

Development Workflow

bash

# Install in development mode (editable)
uv pip install -e ".[dev]"

# Or with pip
pip install -e ".[dev]"

# Run tests
pytest

# Type check
mypy src/

# Lint and format
ruff check src/ tests/
ruff format src/ tests/

# Build the package
uv build
# Or: python -m build

# This creates:
# dist/my_package-0.1.0-py3-none-any.whl
# dist/my_package-0.1.0.tar.gz

Writing a CLI Entry Point

python

# src/my_package/cli.py
import argparse
import sys
from rich.console import Console
from my_package import __version__
from my_package.core import process_data

console = Console()

def main():
    parser = argparse.ArgumentParser(
        prog="my-cli",
        description="A CLI tool for amazing AI things",
    )
    parser.add_argument("--version", action="version", version=f"%(prog)s {__version__}")
    parser.add_argument("input", help="Input file or data path")
    parser.add_argument("--output", "-o", default=None, help="Output file path")
    parser.add_argument("--verbose", "-v", action="store_true", help="Verbose output")

    args = parser.parse_args()

    try:
        result = process_data(args.input)
        if args.output:
            with open(args.output, "w") as f:
                f.write(result)
            console.print(f"[green]Output saved to {args.output}[/green]")
        else:
            console.print(result)
    except FileNotFoundError:
        console.print(f"[red]Error: File not found: {args.input}[/red]")
        sys.exit(1)
    except Exception as e:
        console.print(f"[red]Error: {e}[/red]")
        sys.exit(1)

if __name__ == "__main__":
    main()

Testing Best Practices

python

# tests/test_core.py
import pytest
from my_package.core import process_data, AmazingClient

def test_process_data_basic():
    """Test basic data processing."""
    result = process_data("test_input")
    assert result is not None
    assert isinstance(result, str)

def test_process_data_empty_input():
    """Test empty input handling."""
    with pytest.raises(ValueError, match="Input cannot be empty"):
        process_data("")

@pytest.mark.asyncio
async def test_client_async():
    """Test async client functionality."""
    client = AmazingClient()
    result = await client.process("test")
    assert result.success is True

# tests/conftest.py
import pytest
from my_package.core import AmazingClient

@pytest.fixture
def client():
    """Provide a test client instance."""
    return AmazingClient(base_url="http://localhost:8000")

Use uv for Development

uv is 10-100x faster than pip for package operations:

bash

# Create a virtual environment
uv venv

# Install the package in editable mode
uv pip install -e ".[dev]"

# Add a new dependency
uv add httpx

# Add a dev dependency
uv add --dev pytest

Common Mistakes

• Forgetting __init__.py: Every directory in your package needs one
• Relative imports in scripts: Use absolute imports (from my_package.core import X)
• Hardcoding versions: Use dynamic versioning or importlib.metadata
• Missing py.typed: Add an empty py.typed file for PEP 561 type checking support