Skip to content

Virtual registry: Add cleanup policy workers

What does this MR do and why?

This MR adds background jobs for executing Virtual Registry clean-up policies. Specifically, it:

  1. Adds the Schedulable module to the VirtualRegistries::Cleanup::Policy model to enable scheduling functionality
  2. Implements two new Sidekiq workers:
    • EnqueuePolicyWorker: A cronjob that identifies runnable clean-up policies and enqueues them for execution
    • ExecutePolicyWorker: A limited capacity worker that processes individual clean-up policies (max 2 concurrent jobs)

These changes enable automated execution of clean-up policies according to their configured cadence, allowing users to set up recurring clean-up of registry artifacts without manual intervention. This is the 3rd MR of the Virtual Registry clean-up policy feature, preceded by Add virtual registries cleanup policies model a... (!203532 - merged) and Add execute cleanup policy service for virtual ... (!205440 - merged).

References

Screenshots or screen recordings

N/A

How to set up and validate locally

  1. Create a virtual registry clean-up policy for a group:

    group = Group.all.detect(&:root?)
    policy = VirtualRegistries::Cleanup::Policy.create!(group: group, keep_n_days_after_download: 30, enabled: true).tap { _1.update_column(:next_run_at, Time.current - 1.day) }
  2. Create some test Maven cache entries with old download dates:

    upstream1 = FactoryBot.create(:virtual_registries_packages_maven_upstream, group: group)
    upstream2 = FactoryBot.create(:virtual_registries_packages_maven_upstream, group: group)
    upstream3 = FactoryBot.create(:virtual_registries_container_upstream, group: group)
    
    # stub file upload
    def fixture_file_upload(*args, **kwargs)
      Rack::Test::UploadedFile.new(*args, **kwargs)
    end
    
    old_mvn_entry1 = FactoryBot.create(:virtual_registries_packages_maven_cache_entry, upstream: upstream1, downloaded_at: Time.current - 35.days)
    recent_mvn_entry1 = FactoryBot.create(:virtual_registries_packages_maven_cache_entry, upstream: upstream1, downloaded_at: Time.current - 20.days)
    
    old_mav_entry2 = FactoryBot.create(:virtual_registries_packages_maven_cache_entry, upstream: upstream2, downloaded_at: Time.current - 35.days)
    recent_mvn_entry2 = FactoryBot.create(:virtual_registries_packages_maven_cache_entry, upstream: upstream2, downloaded_at: Time.current - 20.days)
    
    old_container_entry = FactoryBot.create(:virtual_registries_container_cache_entry, upstream: upstream3, downloaded_at: Time.current - 45.days)
    recent_container_entry = FactoryBot.create(:virtual_registries_container_cache_entry, upstream: upstream3, downloaded_at: Time.current - 10.days)
  3. Execute the clean-up worker:

    VirtualRegistries::Cleanup::ExecutePolicyWorker.perform_inline
    => true
    
    policy.reload
    => #<VirtualRegistries::Cleanup::Policy:0x0000000154a17278
    id: 3,
    group_id: 35,
    next_run_at: Wed, 01 Oct 2025 12:44:10.157356000 UTC +00:00,
    last_run_at: Wed, 24 Sep 2025 12:44:10.152147000 UTC +00:00,
    last_run_deleted_size: 3072,
    created_at: Wed, 24 Sep 2025 12:42:50.656135000 UTC +00:00,
    updated_at: Wed, 24 Sep 2025 12:44:10.157609000 UTC +00:00,
    keep_n_days_after_download: 30,
    last_run_deleted_entries_count: 3,
    status: "scheduled",
    cadence: 7,
    enabled: true,
    notify_on_success: false,
    notify_on_failure: false,
    failure_message: nil,
    last_run_detailed_metrics:
     {"maven"=>{"deleted_size"=>2048, "deleted_entries_count"=>2},
      "container"=>{"deleted_size"=>1024, "deleted_entries_count"=>1}}>
    
    upstream1.cache_entries.pending_destruction.size
    => 1
    
    upstream2.cache_entries.pending_destruction.size
    => 1
    
    upstream3.cache_entries.pending_destruction.size
    => 1

💾 Database analysis

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #568107 (closed)

Edited by Moaz Khalifa

Merge request reports

Loading