Skip to content

Draft: Feature/604 pinned requirements

Miron van der Kolk requested to merge feature/604-pinned-requirements into develop

fixes: #604

This is an attempt at creating deterministic, reproducible pipelines.

I'm using pip-tools, which has two commands:

pip-tools compile: This takes our pyproject.toml where the dependencies are specified and creates a requirements.txt-like file that pins the highest possible versions for our dependencies and their dependencies. This file is then only consumed by our pipelines, not by any end-users (setuptools will ignore it as well). If there is an existing requirements.txt that satisfies the requirements in pyproject.toml the pip-tools compile command will do nothing. The requirements.txt should probably be updated by a pre-commit hook every time the pyproject.toml is updated.

pip-tools sync: Takes a requirements.txt and makes sure the current environment exactly matches that. This also includes de-installing packages that are not in the requirements.txt. This is a good way to make sure that the requirements listed are complete.

Conda problems

The above all works fine for the linux pipelines, which do everything with pip. Going to experiment with conda lock

Edited by Miron van der Kolk

Merge request reports