Add an opam lock file and a script to generate/update it
(aims to replace parts of !7026 (closed) eventually)
What
This MR adds a script: update_opam_lock.sh
which, when ran, creates opam/virtual/octez-deps.opam.locked
. This lock file is similar to opam/virtual/octez-deps.opam
except that it contains the transitive closure of Octez dependencies, and all versions are constrained using equalities.
Why
This is a first step towards removing the tezos/opam-repository
machinery.
tezos/opam-repository
is, currently, an opam repository; more precisely, it is a subset of ocaml/opam-repository
which only contains what Octez devs, Octez users and the Octez CI need, at specific version numbers so that everyone use the same dependencies. We believe that using a lock file to specify the version of our dependencies will be simpler than using this machinery, and that it will be more flexible.
For instance, one will be able to update the version of a single dependency by manually editing the lock file. When a dependency needs to be updated just before a release, having to update the world is scary; being able to only update this particular dependency is reassuring. This also allows to update only the dependencies we actually need in release branches, instead of all of them; release branches are supposed to not change too much, and this should also be true for dependencies.
So this MR introduces this lock file, and a way to generate it. In future MRs, we will be able to:
- use this lock file to replace the current implementation of
make build-deps
so that building Octez does not rely ontezos/opam-repository
except in the CI; - change the implementation of
tezos/opam-repository
so that it is not an opam repository (to make the lock file the only source of truth).
How
The script can be summarized as follows:
- tell opam to use a specific commit from the public opam repository, namely
full_opam_repository_tag
fromversion.sh
; - tell opam to install our dependencies;
- tell opam to lock those dependencies into a lock file.
We ask opam to install both Octez dependencies and dev dependencies, so that it finds a solution that is compatible with both. Unfortunately, this has the side-effect of including utop
(and its dependency xdg
) in the lock file, because Octez depends on pyml
, which optionally depends on utop
. And opam lock
take all dependencies, optional or not, that are currently installed. To fix this, we simply run opam remove utop
in the script.
We start from a fresh opam switch so that already-installed packages have no impact, so that we can use --fake
to avoid actually installing the packages, so that we don't have to use --criteria
which would also mean that we would have to remove packages with avoid-version
, and also just so that the script has no impact on the current switch.
Note that the resulting list of packages is not exactly what is in tezos/opam-repository
. See !12029 (comment 1779084724) for a discussion about why this is the case. But the summary is that tezos/opam-repository
contains everything that might be needed by devs and by the CI; some of these are not needed just to build Octez.
Manually testing the MR
scripts/update_opam_lock.sh
git diff
git diff
should return an empty diff.
Checklist
-
Provide automatic testing (see the testing guide). -
Select suitable reviewers using the Reviewers
field below. -
Select as Assignee
the next person who should take action on that MR