Adds a new build-components job that runs automatically for all GitLab.com gitlab-org/gitlab scheduled pipelines, for any master commit that touches the workhorse/ folder, and is manual for GitLab.com's gitlab-org's MRs.
This job tries to download a generic package that contains GitLab Workhorse binaries needed in the GitLab's test suite (under tmp/tests/gitlab-workhorse).
If the package URL returns a 404:
It runs scripts/setup-test-env, so that the GitLab Workhorse binaries are built.
First download the GitLab Workhorse generic package build and uploaded by build-components.
If the package is retrieved successfully, its content is placed in the right folder (i.e. tmp/tests/gitlab-workhorse), preventing the building of the binaries when scripts/setup-test-env is run later on.
If the package URL returns a 404, the behavior doesn't change compared to the current one: the GitLab Workhorse binaries are built as part of scripts/setup-test-env
Notes:
The diff to achieve this is quite small, and we can see that TestEnv didn't have to be touched at all, so this is really an optimization that sits on top of what we have today (i.e. caching of GitLab Workhorse binaries).
The version of the package is the workhorse tree SHA (i.e. git rev-parse HEAD:workhorse).
Since the workhorse shouldn't change too often, most of the MRs should be able to take advantage of the pre-built package, which would be built every two hours.
As a further iteration, the package could be downloaded by TestEnv directly, so that local GDK instances, and GitPod instances (not sure about that) would benefit from it as well (though, different architectures should be taken in account, see gitlab-development-kit#1391).
It looks like we could already save 42 seconds on average for the setup-test-env job.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
