Add a partitioned table for pipeline processing data

What does this MR do and why?

This is the first slice of !253751 (merged), which was too large to review in one piece. It contains only the database groundwork for the redundant-pipeline cancellation work discussed in #621543. Nothing reads or writes the new table yet.

The MR adds:

  • p_ci_pipeline_processing_data, a partitioned table keyed by (pipeline_id, partition_id), with a project_id column and an interruptible_protected boolean.
  • Ci::PipelineProcessingData, and a has_one :pipeline_processing_data association on Ci::Pipeline.
  • A development seed fixture, a factory, a model spec, and entries in all_models.yml and gitlab_loose_foreign_keys.yml.

The table is registered with the partition manager using detach_archived: true, so a partition is detached once every pipeline in it is archived. This is the reason for a new table rather than a column on ci_pipeline_metadata: the data is only useful while a pipeline can still be processed, and ci_pipeline_metadata is not partitioned, keeps every row forever, and is replicated to ClickHouse for analytics that have no use for this column.

The pipeline owns exactly one row, so the pipeline key doubles as the primary key. There is no surrogate id and no extra uniqueness index.

Only the single column this feature needs is added here. Moving the other processing-only columns out of ci_pipeline_metadata is tracked in #607862.

Follow-up MRs will build the Redis-backed redundant-pipeline cancellation on top of this.

No feature flag is needed, because the table is inert.

Database

Two migrations:

  • 20260911170000 creates the partitioned table.
  • 20260911170001 adds a composite foreign key to p_ci_pipelines.

Because the composite foreign key constrains both key columns, the associations do not need a partition scope of their own.

The table is new and empty, so there is no data migration.

How to set up and validate locally

  1. Run the migrations:

    bin/rails db:migrate
  2. Verify the table and its partitions exist:

    gdk psql -c '\d+ p_ci_pipeline_processing_data'
  3. Verify the migrations reverse cleanly:

    bin/rails db:rollback:main STEP=2

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist.

Merge request reports

Loading
Loading