Link Search Menu Expand Document

Pure Python packaging

For Python packages without binary extensions and fairly simple builds can use a modern build system instead of the classic but verbose setuptools. The one you select doesn’t really matter that much; they all use a standard configuration language specified in PEP 621. The PyPA’s Flit is a great option. In the future, scikit-build and meson may support this sort of configuration, enabling binary extension packages to benefit too. These PEP 621 tools currently include Flit, PDM, Trampolim, and Whey. Setuptools might gain support in 2022, and Poetry might as well.

Classic files

These systems do not use or require setup.py, setup.cfg, or MANIFEST.in. Those are for setuptools.

SDist compatibility

Some systems produce a compatibility setup.py to allow use in very old environments, but there is not much need for this - 99% of users of pure Python packages will simply get the wheel - you have to ask for the SDist, and if you do, you are probably a package manager and know what you are doing. And if you are, you need to support modern style builds now; the PyPA packages are starting to drop setup.py’s.

pyproject.toml: build-system

Packages must have a pyproject.toml file that selects the backend (this one is for Flit):

[build-system]
requires = ["flit_core >=3.2"]
build-backend = "flit_core.buildapi"

pyproject.toml: project

The metadata is specified in a standards-based (PEP 621) format:

[project]
name = "package"
description = "A great package."
readme = "README.md"
authors = [
  {name="My Name", email="me@email.com"},
]
maintainers = [
  {name="Scikit-HEP", email="scikit-hep-admins@googlegroups.com"},
]
license = {file="LICENSE"}
requires-python = ">=3.7"

dependencies = [
  "numpy>=1.13.3",
]

classifiers = [
  "Development Status :: 4 - Beta",
  "License :: OSI Approved :: BSD License",
  "Programming Language :: Python :: 3 :: Only",
  "Programming Language :: Python :: 3.7",
  "Programming Language :: Python :: 3.8",
  "Programming Language :: Python :: 3.9",
  "Programming Language :: Python :: 3.10",
  "Topic :: Scientific/Engineering :: Physics",
]

[project.urls]
Source = "https://github.com/scikit-hep/package"
Documentation = "https://package.readthedocs.io/"
"Bug Tracker" = "https://github.com/scikit-hep/package/issues"
Discussions = "https://github.com/scikit-hep/package/discussions"
Changelog = "https://package.readthedocs.io/en/latest/changelog.html"

You can read more about each field, and all allowed fields, in PEP 621, Flit or Whey.

Package structure

All packages should have a src folder, with the package code residing inside it, such as src/<package>/. This may seem like extra hassle; after all, you can type “python” in the main directory and avoid installing it if you don’t have a src folder! However, this is a bad practice, and it causes several common bugs, such as running pytest and getting the local version instead of the installed version - this obviously tends to break if you build parts of the library or if you access package metadata.

This sadly is not part of the standard metadata in [project], so it depends on what backend you you use. Flit and PDM use automatic detection, while Trampolim and whey do not, requiring a tool setting.

Versioning

You can specify the version manually (as shown in the example), but the backends usually provide some automatic features to help you avoid this. Flit will pull this from a file if you ask it to. PDM can be instructed to look in a file or use git. Trampolim can be instructed to use Git.

Including/excluding files in the SDist

This it tool specific. Flit info here. PDM info here. Whey info here.

Extras

It is recommended to use extras instead of or in addition to making requirement files. These extras a) correctly interact with install requires and other built-in tools, b) are available directly when installing via PyPI, and c) are allowed in requirements.txt, install_requires, pyproject.toml, and most other places requirements are passed.

Here is an example of a simple extras:

[project.optional-dependencies]
test = [
  "pytest >=6.0",
]
mpl = [
  "matplotlib >=2.0",
]

Self dependencies can be used by using the name of the package, such as dev = ["package[test,examples]"], but this requires Pip 21.2 or newer. We recommend providing at least test, docs, and dev.