Enable Cleanup policy for tags for all existing projects on both GitLab.com and self-managed instances
Problem to solve
The GitLab Cleanup policy for tags have been rolled out to all new projects, however, it does not cover existing projects. This prevents admin from doing automated clean-up on existing projects and from truly lowering the cost of their container registry.
Why the MVC didn't include all projects
The policy utilizes the Container Registry API to bulk delete tags. That particular API can be slow and has caused issues for customers in the past. Limiting the feature to new projects was done to:
- Limit the risk of the API failing for all of our customers at once
- Get feedback on the feature, for example,
does project-level make sense,
is our default setting sensible? and
are people able to find the feature?
- Roll out performance improvements to the Container Registry delete API, to make them asynchronous.
- Add throttling to prevent bulk delete jobs from overloading the system
- Enable the feature by default for GitLab.com
- Enable the feature by default for self-managed instances
- Alert the user (in the app) when the policies will run
- It's important to note that only instances utilizing the GitLab Container Registry will see the performance improvements. If you are using something like ECR, the jobs will still run, but will generally be slower.
Roll out the Cleanup policy to all existing projects for both self-managed instances and GitLab.com. The feature should default to
on and GitLab self-managed instances to allow users to lower the cost of their container registry.
By default the feature will be enabled for all projects with the below settings:
- The expiration interval is set to 90 days
- The expiration schedule is set to run weekly
- The number of tags to retain is set to 10.
- Expire images matching the regex
.*after 90 days and more than ten matching images
- Preserve images matching the regex
.*master||.*release||release-.*||master-.*)so that master and release images are always preserved.
latestimages are always preserved by default
Permissions and Security
- There are no permissions changes required for this change.
What does success look like, and how can we measure that?
- Success looks like we have rolled this feature out for all self-managed instances and it works reliably and efficiently.
- Number of support issues opened
- Track the total number of projects with expiration policies enabled
- Track the number of projects with expiration interval set to each option
- Track the number of projects with expiration schedule set to each option
- Track the number of tags to retain set to each option