Build git-lfs from source in runner ubi-fips image
What does this MR do?
This MR builds git-lfs from source in the runner ubi-fips docker images.
Why was this MR needed?
https://gitlab.com/gitlab-org/gitlab-runner/-/issues/31065 reports a whole pile of vulnerabilities against the runner:ubi-fips image, many of which are against git-lfs (see https://gitlab.com/gitlab-org/gitlab-runner/-/issues/31065#note_1459094800). The issue specifically is that git-lfs is build with an (old) version of go which confers vulnerabilities onto the git-lfs binary. Since we're already using the latest version of git-lfs (3.3.0 as of July 2023), we have to build it from source, with a new-enough version of go in which the vulnerabilities have been fixed.
Now, why the atrocity that is this MR? The nice way to handle this (and other manually installed dependencies) would be to create a separate docker image that manually installs/builds/whatever the required dependencies in a scratch image. Then downstream images that want to consume these dependencies could just do:
COPY --from=runner-deps /source /destThe problem with this is that runner images exist for a number of architectures and base OS images, so we'd actually need a whole lot of these dependency images. The idea still has merit, but is a much bigger job than the scope of 31065 allows.
Anyway, add to the above that a single script installs all these manual dependencies. This script, install_deps, is parametrised by TARGETPLATFORM to download the right version of the various packages it installs, AND runs in the same image as the final target image. So for example, for runner alpine images, the install_deps scripts run in alpine images, for runner ubuntu images, the script runs in an ubuntu image, etc.
We could try to modify install_deps to build git-lfs from source, but there is a complication: All of these image are of different vintages, and each one packages a different version of Go, in some cases a very very old version of go. Following this approach would fix the immediate issue in the runner uni-fips image, but actually make the matter worse in other images (e.g. older alpine images) since the version of Go package there is old, and has many known vulnerabilities.
...so, rather than tacking all the of the above issues head on, I'm only addressing the immediate problem and simply building git-lfs from source for the runner ubi-fips image only, and doing so in a way that minimally changes the existing infrastructure since I expect this change to be temporary.
The script build_git_lfs does exactly this, and in doing so clobbers the version of git-lfs installed via install_deps. I only feel slightly dirty for this, only because it's a temporary fix, until the next release of git-lfs, which will hopefully be build with a newer version of Go.
What's the best way to test this MR?
Check out the development docker images: [ubi-fips, amd64] pipeline job an confirm this message appears in the logs:
git-lfs/3.3.0 (GitHub; linux amd64; go 1.19.10) (e.g. https://gitlab.com/gitlab-org/gitlab-runner/-/jobs/4615745899#L314)