Add strategy to switch write to new partitions based on the time

What does this MR do and why?

This is to add time-based strategy to switch write to new partitions.

References

Screenshots or screen recordings

Before After

How to set up and validate locally

  • Enable the feature if not yet

    Feature.enable(:ci_time_based_partitioning)
  • Make sure all partitions are created

    Database::PartitionManagementWorker.perform_inline
    Ci::PartitioningWorker.perform_inline
  • Confirm the current partition:

    Ci::Partition.current.id
  • Make sure current partition record has value for current_from

    current_partition = Ci::Partition.current
    current_partition.current_from ||= Time.current
    current_partition.save!
  • Make sure partition records after current partition should be in ready status:

    Ci::Partition.id_after(current_partition.id).update_all(status: Ci::Partition.statuses[:ready])
  • Change settings to time-based strategy:

    settings = ApplicationSetting.first
    settings.ci_partitions_creation_strategy = 'time'
    settings.ci_partitions_time_window = '1 minute'
    settings.save!
  • Run create and sync service:

    Ci::Partitions::SetupDefaultService.new.execute
    
    ci_partition_current = Ci::Partition.current
    return unless ci_partition_current
    
    Ci::Partitions::CreateService.new(ci_partition_current).execute
    Ci::Partitions::SyncService.new(ci_partition_current).execute
  • Check the current partition and see if it's increased by 1

    Ci::Partition.current.id

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #577314

Edited by Tianwen Chen

Merge request reports

Loading