Enable Cleanup policy for tags for all existing projects on both GitLab.com and self-managed instances

Problem to solve

The GitLab Cleanup policy for tags have been rolled out to all new projects, however, it does not cover existing projects. This prevents admin from doing automated clean-up on existing projects and from truly lowering the cost of their container registry.

Intended users

• Sidney (Systems Administrator)

Further details

Why the MVC didn't include all projects

The policy utilizes the Container Registry API to bulk delete tags. That particular API can be slow and has caused issues for customers in the past. Limiting the feature to new projects was done to:

• Limit the risk of the API failing for all of our customers at once
• Get feedback on the feature, for example, does project-level make sense, is our default setting sensible? and are people able to find the feature?

Rollout Plan

• Roll out performance improvements to the Container Registry delete API, to make them asynchronous.
• Add throttling to prevent bulk delete jobs from overloading the system
• Enable the feature by default for GitLab.com
• Enable the feature by default for self-managed instances
• Alert the user (in the app) when the policies will run

Performance gains

• It's important to note that only instances utilizing the GitLab Container Registry will see the performance improvements. If you are using something like ECR, the jobs will still run, but will generally be slower.

Proposal

Roll out the Cleanup policy to all existing projects for both self-managed instances and GitLab.com. The feature should default to on and GitLab self-managed instances to allow users to lower the cost of their container registry.

By default the feature will be enabled for all projects with the below settings:

• The expiration interval is set to 90 days
• The expiration schedule is set to run weekly
• The number of tags to retain is set to 10.

• Expire images matching the regex .* after 90 days and more than ten matching images
• Preserve images matching the regex .*master||.*release||release-.*||master-.*) so that master and release images are always preserved.
• latest images are always preserved by default

Permissions and Security

• There are no permissions changes required for this change.

Documentation

• Expiration Policy docs

Testing

What does success look like, and how can we measure that?

• Success looks like we have rolled this feature out for all self-managed instances and it works reliably and efficiently.

Metrics

• Number of support issues opened

• Track the total number of projects with expiration policies enabled
• Track the number of projects with expiration interval set to each option
• Track the number of projects with expiration schedule set to each option
• Track the number of tags to retain set to each option

Links / references

• Package team meeting to discuss how to roll this feature out to all projects

This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.