Throttling for Container expiration/retention policies
Problem to solve
Currently, the container expiration policy feature is only allowed for projects created after 12.8.
We would like to turn on the feature for all projects, but there are performance concerns because tag deletion can be slow.
When we turn on the policy feature for all projects, we need a way to prevent the sidekiq queue from backing up and slowing things to a halt, especially if there are cases when there are large amounts of tags to be cleaned up for a given policy on it's first run (image we have 10,000 tags, and turn on a policy to keep 100). To prevent thousands of tags from being queued by one project, we should add throttling to the queue or service. This solution needs to be further examined, but for now, see !7292 (merged) for reference on throttling sidekiq queues.
Permissions and Security
Availability & Testing
What does success look like, and how can we measure that?
We are able to enable container expiration and retention policies for all projects on GitLab.com and self-managed instances regardless of which container registry is being used.