GitLab Next

Problem to solve

For organizations with many groups and projects it is more efficient to remove old, unused Docker images utilizing the bulk tag deletion API. The API will:

• Remove only the tags matching the given name_regex.
• Never removes the tag named latest.
• Keep N latest matching tags (if keep_n is specified).
• Only remove tags that are older than X amount of time (if older_than is specified).
• Schedule the asynchronous job to be executed in the background.

However, there is currently no way to express something such as "no matter what, don't delete this tag". This introduces risk into the deletion process, as it's possible to delete release or master images.

Customer quote:

I have many workflows where I have a few tags that periodically move from one image to another (such as "test", "prod" for example, or the branch name), and many immutable tags (usually the git commit sha).

In these cases, I would like to be able to specify a whitelist for tags that must not be deleted, while still cleaning normally the others.

Target audience

• Sidney (Systems Administrator)

Further details

Attempted workarounds

I tried working around this limitation by using a negative lookahead in my regexp, but since the regexp engine used for untrusted regexps is RE2, it is pretty limited. From what I understand, this is by design, for security purposes.

Proposal

• Add an attribute to the bulk delete API, that will prevent any tags that match the provided regex from being deleted.
• Update the name_regex attribute to name_regex_delete. This will remain a required, string attribute that uses the re2 regex of the name to delete.
• Add a new attribute called name_regex_keep. This will be an optional, string attribute that uses the re2 regex of the name to define tags that should not be removed.
• IF the regex for name_regex_delete matches name_regex_keep, nothing will be deleted. Identifying images to protect overrides the delete option.

Permissions and Security

• There are no changes to permissions. The bulk delete API can be utilized by maintainers and owners.

Documentation

• Update the Container Registry API documentation

What does success look like, and how can we measure that?

• We can bulk-delete registry tags while specifying a pattern of tags that must not be deleted.

Metrics

• Count garbage collection commands run
• Count of successful runs
• Count of runs failed
  • Ideally we would capture the error code here and report that as well.

Links