feat(gitlab-runner): enable git object pools on gprd private shards

What does this MR do?

As CI load increases due to agentic workflows (there are developers who are working on 90+ mrs concurrently via agents), we need to be smarter about how we run CI. Currently, every job does its own shallow clone. Though we have mechanisms through the stack to cache similar kinds of fetches, this is still wasteful. The gitlab project should find a way to clone in a central location and share the git data with every job.

This MR opens up a shared mount on the VM that containers where jobs run can share git objects using the git alternates mechanism.

flowchart TB
  subgraph vm["Job VM"]
    c1["Job 1 container"]
    c2["Job 2 container"]
    c3["Job 3 container"]
    pool["/cache/git-object-pools"]
  end
  c1 --- pool
  c2 --- pool
  c3 --- pool
  style vm stroke-dasharray: 5 5

Turn on per-VM git object pools for gitlab-org/gitlab CI on project 1 of the gprd private shards p-s, p-m and p-l (com-gitlab-org, -medium and -large). The managers in that project set one variable in their job environment: GLCI_GIT_OBJECT_POOL_DIR=/cache/git-object-pools. It only says where pools live. The companion gitlab-org/gitlab MR seeds a bare clone of the default branch at $GLCI_GIT_OBJECT_POOL_DIR/gitlab.git on the first job of each VM. Each project's .gitlab-ci.yml picks its own pool name and sets GIT_ALTERNATE_OBJECT_DIRECTORIES itself. It is not set here because these shards run jobs for every gitlab-org project, and a hardcoded gitlab.git alternate would make git print an error on every command in other projects' jobs. Later jobs keep the runner's normal git init and git fetch, so most of each fetch comes from local disk instead of Gitaly. A GLCI_GIT_OBJECT_POOL flag on the project side controls whether jobs seed a pool at all, and is off until this deploys.

The pool lives in the runner's existing per-project /cache volume, which already gets a -protected suffix for protected refs. So other projects can't read the pool, and unprotected pipelines can't tamper with the one protected jobs use. Nothing happens until the companion MR merges, and no other shards or projects change. The variable lives in a clusters/runner-managers-gprd-1/projects/1/values.yaml override, because the shard-level projects file is read before the cluster values.yaml. Helm replaces lists, so the gprd-ci job environment template gets a new shardConfig.extraJobEnv list, appended after jobEnv. The override sets only that. Shards that don't set it render the same config. The other projects follow once the pools look healthy on project 1.

This moves https://gitlab.com/gitlab-com/gl-infra/chef-repo/-/merge_requests/8096 over to argocd now that the runner managers run on Kubernetes. I left out the post_get_sources_script logging from that MR to keep the shared template change minimal.

Companion MR: gitlab-org/gitlab!259066 (merged)

Relevant issues

Related to gitlab-com/gl-infra/production-engineering#29945

🤖 Generated with Claude Code