Manifest: replace ~release with ~release_status
Context
In the manifest, we can declare that an executable is released, with ~release:true
. But there are other sources of truth: the Makefile
and the Dockerfile
. Moreover, some executables are experimental: they should only be "released" for master
, not for actual version tags. Finally, we only want to release opam packages that are needed to build released executables. All of this means that some cleaning up is needed.
This merge request is the first step towards this cleanup. It replaces ~release:true
with ~release_status:Released
, leaving room for ~release_status:Experimental
. One can additionally annotate with ~release_status:Unreleased
to explicitly state that a package must not be released. The manifest will emit an error if a release package depends on an unreleased package. Finally, with ~release_status:Auto_opam
(which is the default), the package is released if, and only if, it is the dependency of another released package.
This merge request introduces function compute_opam_release_graph
which computes the opam dependency graph. Each node contains the release status of the corresponding opam package. This function checks:
- that there is no opam package that was requested to be both released and not released;
- that there is no released opam package that depends on an unreleased opam package.
The resulting graph is used to simplify generate_opam_ci
. Later, the plan is to also use the generated graph to restrict the set of released .opam
files to the set of packages that were annotated with ~release_status:Released
and their transitive dependencies. And the plan is also to generate octez.opam
from the set of packages that were explicitly annotated with ~release_status:Released
. See !6803 (merged) for a first attempt at this.
About opam_package.yml
This file differs from the original because of the rewrite of function generate_opam_ci
. Care should be taken to check that:
- packages that were removed from this list are packages that we do not want to release;
- packages that were added (if any) are packages that we want to release;
- the longest jobs still start sooner than the slowest jobs, i.e. the total pipeline time does not increase.
About binaries-for-release
This file differs from the original because we do not actually want to release executables for protocol Alpha. We do want to publish them in the CI for master
, but note that binaries-for-release
is only used to generate the release at https://gitlab.com/tezos/tezos/-/releases, which is not something we do for master
.
Future Work
In the future, my plan is to build upon this MR as follows:
- rename
binaries-for-release
intoreleased-executables
- introduce
experimental-executables
next toreleased-executables
- use
released-executables
to decide which executables to build withmake
- rename
make build-unreleased
tomake experimental
(keeping the old name as deprecated during a transition period) - use
experimental-executables
to decide which executables to build withmake experimental
- either split
static-packages
intoexperimental-static-packages
+released-static-packages
, or try to use*-executables
directly - use that to only publish experimental static executables for
master
, not for release tags - try to use
*-executables
in theDockerfile
, or generate theDockerfile
, or something, to have only one source of truth, and to only have experimental executables in Docker images formaster
, not release tags - redo !6803 (merged) on top of this
Manually testing the MR
Try to add and remove ~release_status
annotations in packages to see the errors that the manifest gives. For instance, if you add ~release_status:Unreleased
to lib_base
, you get:
Package "tezos-baking-014-PtKathma" is released
because its reverse dependency "tezos-baking-014-PtKathma-commands" is released
because its reverse dependency "octez-accuser-PtKathma" is released
because in src/proto_014_PtKathma/bin_accuser, "octez-accuser-PtKathma" is explicitly released
Package "tezos-base" is not released
because in src/lib_base, "tezos-base" is explicitly not released
Error: Released package "tezos-baking-014-PtKathma" cannot depend on unreleased package "tezos-base".
and if you add ~release_status:Unreleased
to tezos-store.shared
and ~release_status:Released
to tezos-store.unix
, you get:
Error: In src/lib_store/shared, "tezos-store.shared" has release status Unreleased, and in src/lib_store/unix, "tezos-store.unix" has release status Released; those two targets cannot be in the same opam package "tezos-store".
Checklist
-
Document the interface of any function added or modified (see the coding guidelines) -
Provide automatic testing (see the testing guide). -
Select suitable reviewers using the Reviewers
field below. -
Select as Assignee
the next person who should take action on that MR