Introduce Cells claims verification workers and cron scheduling

What does this MR do and why?

  • Add Cells::ClaimsVerificationWorker to run verification per model, gated by a dynamic ops feature flag per model name. This worker calls Cells::Claims::VerificationService introduced in !226233 (merged), to backfill and reconcile changes between Rails and Topology Service.
  • Add Cells::ScheduleClaimsVerificationWorker as a cron job that enqueues a ClaimsVerificationWorker for every model registered via Cells::Claimable with 10 mins delay. This cronjob runs once every Saturdays, and the schedule is randomized between 00:00 to 23:59 to avoid thunderring herd from all cells running this worker. This is done via Gitlab::Scheduling::ScheduleWithinWorker

References

gitlab-com/gl-infra/tenant-scale/cells-infrastructure/team#468

How to set up and validate locally

  1. In console, run Rails.application.eager_load! to ensure all models have been loaded

  2. Enable the feature flags:

     %w[
        cells_claims_verification_worker_organizations_organization
        cells_claims_verification_worker_project
        cells_claims_verification_worker_namespace
        cells_claims_verification_worker_user
        cells_claims_verification_worker_key
        cells_claims_verification_worker_email
        cells_claims_verification_worker_gpg_key
        cells_claims_verification_worker_redirect_route
        cells_claims_verification_worker_route
        cells_claims_verification_worker_service_desk_setting
      ].each { |flag| Feature.enable(flag) }
  3. Run the Cells::ScheduleClaimsVerificationWorker

    Cells::ScheduleClaimsVerificationWorker.new.perform
  4. Check that the claims records are backfilled in topology service database:

    gdk psql -d topology_service
    
    SELECT * FROM claims;

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Marco Gregorius

Merge request reports

Loading