Properly cache gitaly and git binaries in CI
Issue
The real issue is that we weren't archiving the git
binaries properly: !72839 (merged)
Here is a list of files cached for Gitaly:
Initial research (outdated)
When Gitaly started requiring Git v2.33, but our system images used Git v2.31, we ran into a bit of a chicken and egg problem as seen in https://gitlab.com/gitlab-org/gitlab/-/jobs/1704115864#L65. setup-test-env
fails because gitaly-test-build
doesn't build Gitaly because the dir is restored from cache, but we need it to recompile git
so it has the right version.
Since git
wasn't rebuilt, setup-test-env
fails in master because Gitaly wants Git v2.33, not v2.31.
I attempted to force a rebuild in !72823 (closed) if the git
binary were not present. However, when the setup-test-env
cache for Gitaly binaries is restored, it restores a subset of files in tmp/tests/gitaly
. However, this tmp/tests/gitaly
doesn't actually have all the source files, like go.mod
and go.sum
, so any attempts to build will fail.
It seems that we're in a world of pain right now because we're mixing source and binaries in the cache, and it's not easy to disentangle this. Ideally we should call Gitaly's make install
with a PREFIX
, install the binaries to $PREFIX/bin
, and then cache that directory.
For now:
- We have to revert the Gitaly server bump: !72828 (merged)
- We can work around this problem by bumping our build images to use Git v2.33: gitlab-build-images!442 (merged)