<!--IssueSummary start--> <details> <summary> Everyone can contribute. [Help move this issue forward](https://handbook.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows/#contributor-links) while earning points, leveling up and collecting rewards. </summary> - [Collaborate/take over this issue](https://contributors.gitlab.com/manage-issue?action=work&projectId=278964&issueIid=627999) </details> <!--IssueSummary end--> ### Summary `BackfillPipelinesIdRangeOnCiPartitions` (`db/post_migrate/20260512150000_...`, milestone 19.0) aborts with `PG::ExclusionViolation` on `check_ci_partitions_pipelines_id_range_no_overlap` when a `ci_partitions` row between the first and the current partition has no rows in `p_ci_builds`. The migration takes each partition's upper bound from the *next* partition's `MIN(commit_id)`. An empty intermediate partition makes that bound NULL, so a non-current partition is written with an unbounded range, and the current partition's own unbounded range then collides with it. The migration runs in a transaction, so `db:migrate` aborts and every later migration stays pending, regular ones included. Observed on a 19.0.8 upgrade from 18.11.11 (Omnibus, external PostgreSQL, single database). ### Steps to reproduce Three partitions where the middle one has no builds: | `ci_partitions.id` | `status` | rows in `p_ci_builds` | | --- | --- | --- | | 100 | 3 (active) | `commit_id` 1 | | 101 | 3 (active) | none | | 102 | 2 (current) | `commit_id` 2 to 131 | Run `sudo gitlab-rake db:migrate`. ### What is the current *bug* behavior? `db/post_migrate/20260512150000_backfill_pipelines_id_range_on_ci_partitions.rb`: ```ruby Partition.order(:id).to_a.push(Partition.new).each_cons(2) do |partition, next_partition| min = Build.where(partition_id: partition.id).minimum(:commit_id) next unless min upper = Build.where(partition_id: next_partition.id).minimum(:commit_id) partition.update!(pipelines_id_range: min...upper) break if partition.current? end ``` - 100: `min` is 1, `upper` from 101 is NULL, written as `[1,)` - 101: skipped by `next unless min` - 102: last pair member, so `upper` is NULL, written as `[2,)` and rejected `break if partition.current?` is there to keep the unbounded range on the current partition alone, but 100 became unbounded through the NULL upper bound rather than through being current, so the guard never applies. Retrying cannot succeed, because the ranges are recomputed from the same build data on every run. ### What is the expected *correct* behavior? The upper bound should come from the next partition that *has* builds, falling back to unbounded only for the current partition. For the data above: 100 → `[1,2)`, 101 → NULL, 102 → `[2,)`. `Ci::Partitions::SyncService#update_range_id_boundaries` already derives boundaries from the partition's own `MIN`/`MAX` (`min..max`, then `(max.next..)` for the incoming partition), so the migration and the runtime service compute the same column two different ways. ### Relevant logs and/or screenshots ``` main: == 20260512150000 BackfillPipelinesIdRangeOnCiPartitions: migrating =========== rake aborted! StandardError: An error has occurred, this and all later migrations canceled: PG::ExclusionViolation: ERROR: conflicting key value violates exclusion constraint "check_ci_partitions_pipelines_id_range_no_overlap" DETAIL: Key (pipelines_id_range)=([2,)) conflicts with existing key (pipelines_id_range)=([1,)). db/post_migrate/20260512150000_backfill_pipelines_id_range_on_ci_partitions.rb:27:in `block in up' db/post_migrate/20260512150000_backfill_pipelines_id_range_on_ci_partitions.rb:21:in `each_cons' ``` ### Possible fixes Skip empty partitions when resolving the upper bound: ```ruby partitions = Partition.order(:id).to_a mins = partitions.to_h { |p| [p.id, Build.where(partition_id: p.id).minimum(:commit_id)] } partitions.each_with_index do |partition, i| min = mins[partition.id] next unless min upper = partitions[(i + 1)..].lazy.filter_map { |p| mins[p.id] }.first partition.update!(pipelines_id_range: min...upper) break if partition.current? end ``` `spec/migrations/20260512150000_backfill_pipelines_id_range_on_ci_partitions_spec.rb` has no case for an empty partition between two populated ones. Its `skips partitions with no builds` example empties the *first* partition, which does not reproduce this. Any instance where one of the default partitions (100, 101, 102) never received a build hits this on upgrade, so low-CI-usage instances are the likely population. Affected 19.0 through `master`. Distinct from #608165 and #618475, which fix `SyncService` and `PartitionCache` at runtime and leave this migration unchanged. Administrator workaround: write the intended ranges directly, then record the version in `schema_migrations` so the backfill is not retried.