Skip to content

Opt out for manifest GC

Context

The new GitLab Container Registry v2 introduces a new feature for blob and manifest garbage collection (GC).

While beneficial for most use cases, this GC feature poses challenges for build workflows that rely on manifest digests outside the registry, such as pinning images in GitLab-CI or for reproducible local image builds. In scenarios where a manifest is not tagged, it gets cleaned up (after the grace period) by the GC mechanism. In safety-sensitive environments (e.g. like in Siemens), this behaviour impacts reproducibility and auditability of the builds which is not desired.

This proposal was developed in collaboration with @bufferoverflow, @ercan.ucan and @fh1ch .

🛠 with ❤ at Siemens

Proposal

To accommodate use cases requiring manifest persistence (for auditing and reproducibility reasons), we propose introducing an opt-out configuration for manifest GC on a per-project basis. This configuration allows projects to disable manifest GC while still permitting manual cleanup via the manifest DELETE REST API.

The proposed solution involves:

  1. Extending the model Namespace: Add a GC opt-out configuration to the Namespace model in the registry, accessible via new REST API endpoints (GET and PATCH) for managing this setting. This change enables pre-emptive opt-out before image push and ensures compatibility with existing workflows.

  2. Modifying the Manifest GC Worker: Adapt the GC worker to exclude manifests from namespaces with the opt-out flag enabled. This approach requires careful consideration to ensure that disabling the opt-out flag does not overlook manifests previously exempt from deletion and that performance is not adversely affected by a large volume of non-deletable manifests.

Benefits

  • Supports reproducibility and auditability in safety-critical environments, e.g. in industrial and medical sectors.
  • Provides flexibility for projects with specific GC requirements.
  • Ensures registry efficiency without compromising on critical workflow needs.

Open dicsussions

  • Impact on the manifest GC worker's performance?
  • Integration in the GitLab frontend for easy configuration? This could be done later.
Edited by Gerardo Navarro