Draft: Feature/604 pinned requirements
fixes: #604
This is an attempt at creating deterministic, reproducible pipelines.
I'm using pip-tools, which has two commands:
pip-tools compile
: This takes our pyproject.toml
where the dependencies are specified and creates a requirements.txt
-like file that pins the highest possible versions for our dependencies and their dependencies. This file is then only consumed by our pipelines, not by any end-users (setuptools will ignore it as well). If there is an existing requirements.txt
that satisfies the requirements in pyproject.toml
the pip-tools compile
command will do nothing. The requirements.txt
should probably be updated by a pre-commit hook every time the pyproject.toml
is updated.
pip-tools sync
: Takes a requirements.txt
and makes sure the current environment exactly matches that. This also includes de-installing packages that are not in the requirements.txt
. This is a good way to make sure that the requirements listed are complete.
Conda problems
The above all works fine for the linux pipelines, which do everything with pip
. Going to experiment with conda lock