Add an automated task to upload archive of master
This commit adds a cache-repo step in the sync stage of the CI
build. It builds a cached .tar.gz of the master branch with full history
so that the deltas and uploads it to Google Cloud Storage. This archive
is downloaded by a script defined by a CI/CD variable named
CI_PRE_CLONE_SCRIPT. This has two benefits:
- It speeds up builds. A 800 MB download only takes seconds.
- It significantly reduces load on the file server. Smaller deltas means less time spent in git pack-objects.
Since the destination directory of the archive depends on the project ID, this is only run on GitLab.com.
CI_REPO_CACHE_CREDENTIALS contains the Google Cloud service account JSON
for uploading to the gitlab-ci-git-repo-cache bucket. These credentials
are stored in the Production vault.
Note that this bucket should be located in the same continent as the runner, or network egress charges will apply: https://cloud.google.com/storage/pricing
Example successful run: https://gitlab.com/stanhu/gitlab/-/jobs/377401042
Closes #39134 (closed)