Skip to content

Implement automated cleanup policy for container image tags

Background

The runwayctl repository currently creates container images and tags on each merge to main. We have accumulated three container images with over 3000 tags each, leading to unnecessary storage consumption and registry clutter.

Problem Statement

Without a cleanup policy, the number of image tags will continue to grow indefinitely, resulting in:

  • Increased storage costs
  • Slower registry operations
  • Difficulty in finding relevant tags
  • Potential performance degradation

Proposed Solution

Implement an automated cleanup policy for container image tags with the following retention rules:

Retention Policy

  • Keep indefinitely:

    • Semantic version tags (e.g., v1.2.3, v2.0.0)
    • Truncated semantic tags (e.g., v1.2, v1)
    • Latest tag

  • Expire after 30 days:

    • Development/feature branch tags
    • Commit SHA tags
    • Timestamp-based tags
    • Any other non-semantic tags

Implementation Tasks

  • Evaluate GitLab's native features

    • Check if GitLab's built-in expiration policies meet our requirements
    • Review container registry cleanup policies in project settings
    • Document findings and limitations

  • Research existing GitLab internal solutions

    • Search GitLab handbook for container registry cleanup practices
    • Check other GitLab team repositories for similar implementations
    • Review gitlab-org namespace projects for cleanup patterns
    • Reach out to #infrastructure or #production Slack channels for recommendations
    • Check if Platform/Infrastructure teams have a standard solution
    • Document any reusable components or patterns found

  • Implementation (based on research findings):

    • If native GitLab solution is sufficient:
      • Configure cleanup policy via GitLab UI or API
      • Test policy on a non-production registry
    • If existing internal solution found:
      • Adapt and integrate the solution to our repository
      • Credit the original implementation team
    • If custom solution needed:
      • Implement cleanup job in main branch pipeline
      • Use GitLab Container Registry API for tag management
      • Add appropriate logging and dry-run capability
      • Consider contributing solution back for other teams to use

  • Documentation:

    • Update repository documentation with cleanup policy details
    • Add monitoring/alerting for cleanup job failures (if applicable)
    • If creating new solution, document for other teams' benefit

  • Testing:

    • Verify semantic tags are preserved
    • Confirm non-semantic tags are removed after 30 days
    • Test edge cases (e.g., recently pushed tags, tags in use)

Acceptance Criteria

  • Automated cleanup process is active and running
  • Semantic version tags remain untouched
  • Non-semantic tags older than 30 days are automatically removed
  • Process is documented and monitored
  • No disruption to existing CI/CD workflows
  • Solution follows GitLab internal best practices (if they exist)

References

Additional Notes

Investigation priority:

  1. Check for existing internal GitLab team solutions (most efficient if available)
  2. Evaluate GitLab's native cleanup policy
Edited by Silvester Wainaina