Building a Python extension with a local C++ library dependency using setuptools and build

I finally came up with a solution, although there are some reasons to consider other options: According to PEP 517, the source tree should be self-contained, so one could argue that the source code of the C++ library should be included in it. I did not want to change the directory structure or add another repository for the Python package, though. Apprently using a build backend such as scikit-build would have made it possible (easier?) to use CMake as part of the build system instead of trying to build everything with setuptools, but I did not want to switch to that either.

The basic idea is to:

• (Ab)use Python’s build module’s configuration options to pass the current directory to the build system with a command similar to python -m build -C project-root=`pwd`. Running the command can be made easier by writing it to a Makefile.
• Write a build backend based on setuptools.build_meta that handles the configuration option.
• Write a build command based on setuptools.command.build_ext.build_ext that modifies relative header and library paths.

Here is the updated directory structure.

python
├── Makefile
├── pyproject.toml
├── README.md
├── my-python-package
│   ├── __init__.py
│   ├── _custom_build.py # Build backend and command
│   ├── py.typed
│   └── my-python-module.pyi
└── src
    └── my-python-module.cc

Here are the relevant parts of pyproject.toml. I used relative paths for include-dirs and library-dirs under [[tool.setuptools.ext-modules]].

[build-system]
requires = ["setuptools >=80.9"] # An older version might have sufficed.
build-backend = "_custom_build"
backend-path = ["my-python-package"]

[tool.setuptools]
packages = ["my-python-package"]

[tool.setuptools.cmdclass]
build_ext = "_custom_build.build_ext"

Here is _custom_build.py.

from setuptools.build_meta import *
from setuptools.build_meta import build_wheel as build_wheel_
from setuptools.command.build_ext import build_ext as build_ext_
import typing


project_root: str = ""


def fix_path(path: str) -> str:
    if path.startswith("/"):
        return path
    return f"{project_root}/{path}"


def build_wheel(wheel_directory, config_settings = None, metadata_directory = None):
    global project_root
    if config_settings:
        project_root = config_settings.get("project-root", "")
    return build_wheel_(wheel_directory, config_settings, metadata_directory)


class build_ext(build_ext_):
    @typing.override
    def initialize_options(self):
        super().initialize_options()
        # Add the include and library paths outside the build directory,
        # since the library is built separately.
        for ext in self.distribution.ext_modules:
            ext.include_dirs = list(map(fix_path, ext.include_dirs))
            ext.library_dirs = list(map(fix_path, ext.library_dirs))