Enable detaching archived partitions of job_definitions & job_definition_instances tables

What does this MR do and why?

Context

We want to reclaim disk by detaching, then dropping, the archived partitions of p_ci_job_definitions and p_ci_job_definition_instances (#552078).

In earlier MRs on that issue we merged the shared machinery this depends on: DetachEligibility refuses a detach that Postgres would reject outright or that would force a scan of a referencing table, and PartitionManager supports detaching concurrently as a per-table opt-in. Neither table is set up to detach anything yet. This MR does that.

This MR

  • Ci::Partitionable.partitionable gains a detach_archived: option, default false, turned on only for Ci::JobDefinition and Ci::JobDefinitionInstance; no other CI partitionable model changes behavior.
  • Ci::Partition gains a predicate for whether every value in a set is archived, and CiSlidingListStrategy#extra_partitions returns the oldest contiguous run of partitions that predicate accepts, never the most recent one.
  • We set detach_concurrently: true for all CI partitionable models. It stays inert for a model that never detaches, since PartitionManager only reads it once there are partitions to detach.
  • The whole path sits behind ci_detach_archived_partitions (gitlab_com_derisk, default off, :instance actor), so nothing detaches until it's enabled.

Feature Flag: ci_detach_archived_partitions

References

Database queries

See MR comments.

How to set up and validate locally

  1. Run the specs:

    bundle exec rspec spec/models/concerns/ci/partitionable_spec.rb \
      spec/models/ci/partition_spec.rb \
      spec/lib/gitlab/database/partitioning/ci_sliding_list_strategy_spec.rb
  2. In a Rails console, archive a partition that is not the current one and check the predicate with the flag on, then off:

    Feature.enable(:ci_detach_archived_partitions)
    
    strategy = Ci::JobDefinition.partitioning_strategy
    database_partition = Gitlab::Database::Partitioning::MultipleNumericListPartition
      .new(:p_ci_job_definitions, [100])
    
    Ci::Partition.find(100).update!(status: Ci::Partition.statuses[:archived])
    strategy.detach_partition_if.call(database_partition)   # => true
    
    Feature.disable(:ci_detach_archived_partitions)
    strategy.detach_partition_if.call(database_partition)   # => false
    
    Ci::Partition.find(100).update!(status: Ci::Partition.statuses[:active])

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Leaminn Ma

Merge request reports

Loading
Loading