Skip to content

Pin auxiliary binaries the main Gitaly binary invokes

Sami Hiltunen requested to merge smh-pack-aux-binaries into master

We have to maintain backwards compatibility between Gitaly and its auxiliary binaries as they may originate from different builds during an upgrade. This has caused incidents in the past and slows down iteration velocity. To avoid the incidents and th burden of backwards compatiblity, it would be great if we could guarantee Gitaly only ever invokes the auxiliary binaries from the same build. We now have the infrastructure in place to locate binaries on disk and to pack the auxiliary binaries in the main Gitaly binary to ship them as a single deployment. This commit hooks wires up the final changes to ensure the correct binaries get invoked always.

The Gitaly binary contains the auxiliary binaries packed into itself. On start up, Gitaly will unpack the binaries into the runtime directory. BinaryPath helper used to locate binaries is updated to correctly locate the unpacked binaries from the runtime directory. Since each Gitaly process has its own runtime directory, the upgrades do not affect the binaries unpacked there. Replacing the gitaly binary on the disk itself is atomic. The combination ensures that the correct binaries are always invoked.

The testing infrastructure is updated to also locate the packed binaries to the runtime directory. The auxiliary binaries are not always built prior to tests but as a part of them. Most of the tests thus do not exercise the packing or unpacking but only the fact that certain binaries are used from the runtime directory.

The packed binaries do not need to be installed anymore so the Makefile is updated not to do so.

Closes #4303 (closed)
Closes #3253 (closed)

Merge request reports