Swap bigint columns for ci_sources_pipelines

What does this MR do and why?

This is the swap step of converting the ci_sources_pipelines columns id, project_id, and source_project_id from integer to bigint. It swaps the columns with their *_convert_to_bigint counterparts.

It adds three post-deployment migrations, all milestone 19.4, and they must run in this order:

  1. 20260831065014_add_tmp_not_null_check_on_ci_sources_pipelines_project_id.rb
  2. 20260831065015_validate_tmp_not_null_check_on_ci_sources_pipelines_project_id.rb
  3. 20260831065016_swap_columns_for_ci_sources_pipelines_bigint_conversion.rb

What the swap does

Before the transaction, it recreates the bigint primary key index with the add_bigint_column_indexes helper, in case that index was dropped. This is the same helper the earlier index-duplication migration used, so the name and options match. The call is idempotent and skips creation when the index already exists.

Inside a lock-retried transaction it:

  • swaps each integer column with its *_convert_to_bigint counterpart
  • swaps the column defaults
  • resets the trigger functions
  • swaps the primary key constraint ci_sources_pipelines_pkey
  • swaps the indexes index_ci_sources_pipelines_on_project_id and index_ci_sources_pipelines_on_source_project_id
  • exchanges the names of the two project_id NOT NULL checks

Everything in the transaction is a catalog operation. No table is rewritten and no constraint is validated there.

There are no foreign keys on the converted columns, and no foreign keys from other tables point at this table's id. So there is no foreign key swap.

db/structure.sql is not changed on purpose. On new installations these columns are already bigint, so skip_bigint_migration? returns true and the migrations do nothing.

The project_id NOT NULL check

project_id is nullable at the column level. Its NOT NULL guarantee comes from CONSTRAINT check_5a76e457e6 CHECK ((project_id IS NOT NULL)).

A check constraint stays on the physical column, so a plain swap would leave it guarding the retired integer column, and the cleanup step would drop it with that column. The sharding key would lose its guarantee for good.

The first two migrations make the bigint column match the integer column, the same way the process already handles indexes and foreign keys: check_5a76e457e6_tmp is added on project_id_convert_to_bigint as NOT VALID, then validated. The swap then only exchanges the two constraint names, so check_5a76e457e6 ends up on the new bigint project_id and the retired column carries the temporary name, which the cleanup step drops.

The check is already valid when the swap runs, so the sharding key is never guarded by a NOT VALID constraint. Exchanging two names is its own inverse, so down reverses it with no extra work.

This is a gap in the conversion tooling, not something specific to this table. BigintConverter mirrors only column-level NOT NULL, so 27 pending conversions have the same exposure. Tracked in #623184.

Ordering

The bigint indexes must exist in production before the swap runs. They shipped in !242049 (merged), which merged for 19.3. This swap targets 19.4, so the indexes are already deployed.

The NOT NULL check must also exist and be valid before the swap renames it. Post-deployment migrations run in timestamp order, so the two prep migrations run first in the same deploy.

Database testing

Run on 2a3b70d5, all three databases pass.

Migration main ci sec
065014 add the check 4.0 s 4.7 s 4.6 s
065015 validate it 3.1 s 147.7 s 4.1 s
065016 swap 9.7 s 10.0 s 11.1 s

The swap runs 18 statements on the ci database, all catalog work, the longest 10.1 ms. There is no ADD CONSTRAINT and no VALIDATE among them.

The validation is flagged for exceeding the 100 ms query guideline, at 143.6 s on ci. That is the full table scan, which any VALIDATE CONSTRAINT on a table this size will hit. It holds SHARE UPDATE EXCLUSIVE, so reads and writes are not blocked, and it sits in its own migration ahead of the swap.

The reported database size change for the swap on ci is -5.38 GiB, consistent with the earlier runs on this branch.

Rollback

down was exercised by hand against a local ci database, since the testing pipeline only runs up. The pre-swap state was rebuilt (the three columns as integer, the three *_convert_to_bigint columns, a sync trigger, the three bigint indexes, one row), then all three migrations were run up and down again.

A dump of columns with types, nullability and defaults, every index definition, every constraint with its convalidated flag, triggers, sequence ownership and the row is identical before and after. The bigint primary key index is recreated, and check_5a76e457e6 returns to the integer column.

Edited by Oleg Yakovenko

Merge request reports

Loading
Loading