Add the Artifact Registry credential to the CI job payload (monolith/S16 plan: 1/6)

What does this MR do and why?

A CI job that names an Artifact Registry image in image: or services: fails its pre-job pull today, because the runner has no credential for the Artifact Registry host. Users have to build a DOCKER_AUTH_CONFIG by hand.

This MR adds an Artifact Registry registry credential to the job payload, so the runner pulls those images with the job user's identity. It is Step 1 of the monolith/S16 plan, as monolith/S16 defines it, behind the ci_artifact_registry_pull_credentials beta flag (off by default, actor: the job's project).

  • Extension point: Gitlab::Ci::Build::Credentials::Factory gains prepend_mod, and its EE override appends Registry::ArtifactRegistry to the providers. The provider has the shape of the existing two, so the payload serializer and existing runners need no change.
  • Conditions, in-memory first: Artifact Registry configured, flag on, the requesting runner manager reports a known executor other than Kubernetes; then two indexed reads: the organization has activated Artifact Registry (has_one on a unique index), and the job's user is a member of the project's organization (exists? on the unique (organization_id, user_id) index, memoized per request). A job skipped in memory pays no query.
  • Kubernetes is excluded because that executor writes payload credentials into a Secret in the build namespace, which other workloads there can read.
  • Token: Authn::TokenExchange::TokenIssuer for the job's user and the project's organization, audiences gitlab-artifact-registry and gitlab-iam-data-access, no gitlab.job claims. It expires at the job's start plus its timeout, at most 12 hours (the ADR-020 maximum).
  • Errors: the token is issued inside valid? and memoized, so an error stays inside the provider. Any error after the in-memory checks is logged with log_exception and counted, and the job runs without the credential. It is not sent to Sentry, which sends in the calling thread inside the present_build phase timer, and whose rescue would swallow the phase timeout; Verify's pre-assign path logs only for the same reason. Ci::RegisterJobService::PhaseTimeoutError is re-raised, so a request that runs out of time still drops the job with scheduler_failure.
  • Job Router: that endpoint serializes the job a second time after present_build!. Ci::BuildRunnerPresenter#credentials is memoized, so both serializations share one provider and one token.
  • Counter: gitlab_ci_artifact_registry_pull_credentials_total{result, reason}, with result issued or skipped and a fixed reason set, documented in gitlab_metrics.md.

Database

The provider adds three reads to the job request path, and a fourth when it issues the token, only for a job in a project with the flag on, on a runner that reports a known executor other than Kubernetes. A job skipped by those in-memory conditions runs none of them.

-- project.organization (primary key)
SELECT "organizations".* FROM "organizations" WHERE "organizations"."id" = $1 LIMIT 1;

-- organization.artifact_registry_activated? (unique index index_artifact_registry_namespace_mappings_on_organization_id)
SELECT "artifact_registry_namespace_mappings".* FROM "artifact_registry_namespace_mappings"
WHERE "artifact_registry_namespace_mappings"."organization_id" = $1 LIMIT 1;

-- user.member_of_organization? (unique index index_organization_users_on_organization_id_and_user_id; memoized per request)
SELECT 1 AS one FROM "organization_users"
WHERE "organization_users"."user_id" = $1 AND "organization_users"."organization_id" = $2 LIMIT 1;

-- TokenIssuer's organization_role claim, user.owns_organization? (index_organization_users_on_organization_id_and_user_id; memoized per request), only for an issued credential
SELECT 1 AS one FROM "organization_users"
WHERE "organization_users"."user_id" = $1 AND "organization_users"."organization_id" = $2
  AND "organization_users"."access_level" = 50 LIMIT 1;

They run in present_build!, after build.run! has written in the same request, so load balancing sends them to the primary. Each is a single-row lookup on a primary key or unique index. Query plans from Database Lab (gitlab-production-main):

  1. organizations by primary key: https://postgres.ai/console/gitlab/gitlab-production-main/sessions/82373/commands/189706
  2. artifact_registry_namespace_mappings by organization_id: https://postgres.ai/console/gitlab/gitlab-production-main/sessions/82373/commands/189707
  3. organization_users by (organization_id, user_id): https://postgres.ai/console/gitlab/gitlab-production-main/sessions/82373/commands/189708
  4. organization_users owner check: https://postgres.ai/console/gitlab/gitlab-production-main/sessions/82373/commands/189713

References

Screenshots or screen recordings

Not applicable: no UI change.

How to set up and validate locally

  1. Run the specs:

    bin/rspec ee/spec/lib/gitlab/ci/build/credentials/registry/artifact_registry_spec.rb \
      ee/spec/requests/api/ci/runner/jobs_request_post_spec.rb \
      ee/spec/requests/api/internal/ci/job_router_artifact_registry_credential_spec.rb \
      spec/presenters/ci/build_runner_presenter_spec.rb \
      spec/lib/gitlab/ci/build/credentials
  2. With Artifact Registry configured (artifact_registry.api_url in gitlab.yml), an activated organization and a Cloud Connector key, enable the flag for a project:

    Feature.enable(:ci_artifact_registry_pull_credentials, Project.find_by_full_path('<path>'))
  3. Run a job on a Docker executor runner whose image: names an Artifact Registry image, with no DOCKER_AUTH_CONFIG. The pull succeeds, and gitlab_ci_artifact_registry_pull_credentials_total{result="issued"} increases.

One spec note: the phase-timeout request spec raises the query threshold to 125. Dropping a job takes about 110 queries in that setup with or without the credential, over the default of 100; the spec says so in a comment.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

🤖 Generated with Claude Code

Edited by Daniel Prause

Merge request reports

Loading
Loading