While working on the contributing documentation for many Plone projects and packages, I noticed that many use GNU make as the tool to wrap commands to build a development environment, run code quality checks and formatting, run tests, and other routine development tasks. Many use tox. There may be other tools, such as one-off shell or Python scripts, that don't have a wrapper.
I think it is a disservice to developers not to unify the tools that they use to develop Plone. I know there is an effort with Plone meta to change that, where tox is used but make is omitted.
However, I found that using tox to build documentation for plone.api was significantly slower than using make, after the initial installation and build, so much so that it made me dislike tox. That might be because I don't know how to properly configure the tox environment for docs. For me, make is easier to configure because I can use whatever the tool provider recommends for environment variables and configuration options, instead of looking up in the tox docs the appropriate flag or option to use. Tox feels like it has a level of indirection that I don't want.
On the other hand, tox makes it vastly easier to test across multiple Python versions. I don't know how to do with with make, other than wrap tox with a make command. For the sake of unified commands, that might be a good way to go.
Are there any advantages or disadvantages to using make versus tox (or any other tool)?
tox is an environment orchestrator. Use it to define how to setup and execute various tools on your projects. The tool can set up environments for and invoke:
Is there some way to tell tox, "Hey, I already installed the package as an editable package and Sphinx will use that, so you, dear tox, don't need to force reinstall things every time!". Make won't run installation steps after the first time they are run.
# create and setup environment "plone6docs" and install the required packages
python3 -m tox run -e plone6docs
# check the installed packages in the environment "plone6docs"
./.tox/plone6docs/bin/pip freeze
...
# run the environment "plone6docs" skipping package install for this run
python3 -m tox run -e plone6docs --skip-pkg-install
The following will create and setup an environment plone6docs-install with skip_install = False
python3 -m tox run -e plone6docs-install
The following will create and setup an environment plone6docs-skip with skip_install = True. In the first run though you must override the option skip_install = False. Otherwise it will fail because the required package is not yet installed.
python3 -m tox run -e plone6docs-skip --override testenv:plone6docs-skip.skip_install=False
python3 -m tox run -e plone6docs-skip
Remember that the feature Generative environments generates separate environments. In the example above the environments plone6docs-skip and plone6docs-install.
I have a hard time remembering options, but the two commands *-install and *-skip work better for me.
Makefile targets whose names are files or directories will only run if they do not exist on the file system. This is the magic of make. For example, if you have a Makefile target like this:
It runs if and only if there is no bin/python, or Python virtual environment. This step would be skipped if bin/python exists.
I don't know of anything like that in tox, other than writing a shell command that checks for existence and runs the set up commands accordingly. Something like the following.
Programming languages, either declarative (like make) or procedural (like bash) are not conceptually in the same paradigm as tools like tox.
Tox is not a programming language, it's an environment orchestrator. It includes the possibility to run commands and other functionalities. But tox allways creates and runs a defined environment. Once the environment is set it invokes whatever commands and tools you want.
From a tox perspective invoking a command like the following in a given tox environment can be rephrased as: create an environment (as tox environment) that invokes the creation of another environment (as environment command).
Tox won't check if a directory or a file exists. Tox will make sure that a given environment is set as defined, with the defined python version and the defined requirements. And the different tox options allow to configure which parts should be recreated or skipped. This is the magic of tox.
The model of make (or bash) is a file system. It sees files and directories.
The model of tox is an environment. It sees python and pip versions and packages.
Tox and make (or bash) cannot be compared, they serve different purposes. They can be combined to fulfill the needs of a project. So we get a lot of magic to deal with.
AFAICS the purpose of the environment plone6docs in plone.api is basically to set an environment to run sphinx-build.
One possible use case is that one might want to make changes on the documentation and run sphinx-build again to see the built documentation. In this case we could first run tox run -e plone6docs to set up the environment and build the sphinx documentation. That way we have the environment (in ./tox/plone6docs) with an editable wheel and the documentation built in the target directory.
In this environment we can simply run the following command every time we change some file in the docs without the need of recreate the whole environment:
.tox/plone6docs/bin/sphinx-build -b html -d _build/plone6docs/doctrees docs _build/plone6docs/html
Sorry, but that command is not simple to me. I could wrap it in make with make html, for example, to make it a simple and memorable command. Nonetheless, using the .tox directory with make had not occurred to me, so thanks for the idea! It's so obvious now.
I see the following common use cases with docs. First, I assume that we do not need to build docs for multiple Python environments, one of tox's advantages. One environment is good enough.
Create the Python virtual environment and install dependencies: make docs-install. This would always include Sphinx, and when the package's API gets documented, then install the package, too.
Build docs with an existing environment: make docs-html.
Clean docs build directory, because Sphinx writes only changed files and does not account for rewriting navigation on unchanged files: make docs-clean.
Add a new dependency, whether that is by me or another developer. For this one, make won't work well unless I specify the dependency in the make command, which is unmaintainable. I would have to delete the environment and recreate it with make docs-env-clean docs-install. I think that tox may be better suited for this task, in that it would not delete the environment and recreate it, but it would add the dependency automagically. Is my thought correct here? If so, that would be very useful.
As a side note, I have a plan to create a cookiecutter template that generates a basic documentation environment for Plone projects with all the essential bits and pieces. I want to figure out these details before I start on that project.
To confuse everyone, here my way to deal with sphinx docs: use mxmake.
pip install mxmake
Create a folder and enter it (commandline)
mxmake init (now interactive)
Selections:
Topics: select only docs,
Domains: select only docs.sphinxdoc,
now just press enter for all the questions (use defaults)
create a folder mkdir -p docs/source and create or copy all your sphinx source (including conf.py and text files) there.
edit Makefile, in the settings part find variable DOCS_REQUIREMENTS and add all sphinx extensions and themes (as defined in conf.py). there, space separated, i.e. DOCS_REQUIREMENTS?=myst_parser sphinx_copybutton sphinxcontrib.mermaid furo
run make docs
For development there is make docs-live.
This can be combined with all other features of mxmake, like managing a full Python package for a backend and so on.
I agree: for the docs only one environment is necessary. That environment must ideally have everything necessary to manage, test, build, and deploy the docs.
As for the use cases:
On the one side we have to make sure that we have a defined environment that can easily be created, setup and run. This environment should ideally automate tests and deployment. And we should be able to modify the definition of the environment. For these tasks we need a environment management tool (from simply venv/pip to more complex tools like tox, nox, hatch etc.). Tox is well known and it is the de-facto standard and it offers everything we need to create the environment, install dependencies, etc. (your use case #1).
On the other side we want to work in the above described environment. Here we use the tools that are installed in the environment (e.g. sphinx). Since the tools have its CLI we can use them without the need of make or any other tools. (your ues cases #2, #3, and some aspects of #4).
Furthermore you might need to add a new dependency (use case #4). This might be ambiguous: It can mean to add a new dependecy to the definition of the environment. For this case we would use tox deps (e.g. with a requirements file). Or it can mean to add a new dependency to the actually created environment (not to the tox definition) for whatever purpose (e.g. test a sphinx extension). For this case we would simply pip install a package in the environment and not modify the definition of the environment. For it we won't need tox but the tools of the environment.
I think it is very important to differentiate the use cases in "managing the environment" on the one side and "working in an environment" on the other side. Once we make the distinction clear we can think about make the developer's life easier. E.g. wrapping the needed commands with aliases, bash functions, make scripts or (more pythonic) in Invoke tasks.
Such wrappings though become inevitable opinionated and even if they make the work easier they always obfuscate the real functionality and prevent the developer to see what he/she is actually doing. For that reason I tend to not use them and I prefer to call the actual commands which I usually find described in my notes (that is what I meant when I wrote "simply run the command". The command and its options might be complex but e.g. copy and paste it to a shell is really simple). But this is my personal opinion.
If we do want to wrap the commands, I would recommend to make clear which commands do manage the environment (tox commands) and which ones do use commands in the environment (e.g. sphinx commands).
For wrapping commands I would avoid extra requirements (like make) that might prevent the developer to work in a machine that doesn't have the necessary tools installed. The pythonic alternative here is Invoke.
For combining both (managing enviromentsand scripting in enviroments) there are other tools like Nox or Hatch.
Here is an example of how to put it together with Invoke.
# clone plone.api
git clone https://github.com/plone/plone.api.git
cd plone.api
# create a venv and pip install tox and invoke
python3 -m venv .venv
.venv/bin/pip install tox invoke
touch tasks.py
copy the following into file tasks.py:
from invoke import task
import os
ENVNAME = "plone6docs"
SPHINX_SOURCE_DIR = "docs"
SPHINX_DOCTREE_DIR = f"_build/{ENVNAME}/doctrees"
SPHINX_OUTPUT_DIR = f"_build/{ENVNAME}/html"
TOX_CMD = ".venv/bin/tox"
@task(help={'recreate': "adds tox option --recreate"})
def create_env(c, recreate=False):
"""
Create the Python virtual environment and install dependencies
"""
c.run(f"{TOX_CMD} run -e {ENVNAME} {'--recreate' if recreate else ''}")
@task(help={'clean': "invoke task clean-docs before build docs"})
def build_docs(c, clean=False):
"""
Build docs with the created environment
"""
if clean:
clean_docs(c)
if not os.path.exists(f".tox/{ENVNAME}/bin/sphinx-build"):
create_env(c, recreate=True)
c.run(f".tox/{ENVNAME}/bin/sphinx-build -b html -d {SPHINX_DOCTREE_DIR} {SPHINX_SOURCE_DIR} {SPHINX_OUTPUT_DIR}")
@task
def clean_docs(c):
"""
Clean docs build directories
"""
c.run(f"rm -rf {SPHINX_DOCTREE_DIR}; rm -rf {SPHINX_OUTPUT_DIR}")
List available tasks and show help of task build-docs:
$ .venv/bin/invoke --list
Available tasks:
build-docs Build docs with the created environment
clean-docs Clean docs build directories
create-env Create the Python virtual environment and install dependencies
$ .venv/bin/invoke --help build-docs
Usage: inv[oke] [--core-opts] build-docs [--options] [other tasks here ...]
Docstring:
Build docs with the created environment
Options:
-c, --clean invoke task clean-docs before build docs
Run as follows:
.venv/bin/invoke create-env --recreate build-docs --clean
#or
.venv/bin/invoke create-env --recreate
.venv/bin/invoke build-docs --clean
... # make some changes in your docs
.venv/bin/invoke build-docs
... # make some changes in your docs
.venv/bin/invoke build-docs