Skip to content

GC: Track temporary manifest blobs

João Pereira requested to merge gc-track-tmp-manifest-blobs into master

As explained in gitlab-com/www-gitlab-com!60918 (comment 409812453), during the first phase of the metadata database rollout we'll continue to write metadata (including manifests, saved as blobs) to the storage backend for backward compatibility. Once we stop saving manifest blobs to the storage backend we need a way to delete the ones that were saved there during the first phase of the rollout (we have the manifest payloads on the DB, so there is no reason to keep a copy of them in the storage backend beyond a backward compatibility guarantee for a limited amount of time).

For this reason, we have a temporary table, named gc_tmp_blobs_manifests, where a row is created for each created manifest. This table is fed by a trigger for inserts on manifests. When the time comes we can extract the list of manifest blobs (identified by digest) from this table and delete them from the storage backend. This entity will then be dropped from the database.

Edited by João Pereira

Merge request reports