Route LFK deleted records by sharding key for three pilot tables

What does this MR do and why?

Phase 3b of the loose foreign keys Cells work: rewrites the LFK trigger of three pilot tables so deleted rows are routed to the sharding-key deleted-records tables instead of the cell-local one. First production use of the routing helpers introduced in Phase 3a.

Pilot Database Sharding keys What it exercises Table Activity (7 days)
project_repositories main project_id (NOT NULL) Base case, track_record_deletions_with_sharding_keys https://dashboards.gitlab.net/goto/afwy3hxfpsohsc?orgId=1, https://dashboards.gitlab.net/goto/ffwy3vyxzjpq8f?orgId=1
p_ci_workloads ci project_id (NOT NULL) Partitioned table, override_table_name variant, ci copy of the sharded tables https://dashboards.gitlab.net/goto/bfwy3ni4c4bnkd?orgId=1, https://dashboards.gitlab.net/goto/bfwy3yqda1urka?orgId=1
clusters main project_id, group_id, organization_id (num_nonnulls = 1) Multi-target dispatch, and source column (group_id) different from target column (namespace_id) https://dashboards.gitlab.net/goto/efwy4mbtt6874f?orgId=1, https://dashboards.gitlab.net/goto/bfwy4mp7u2ubkb?orgId=1

All three have exclusive, guaranteed-non-null keys, so every deleted row produces exactly one routed record: no fan-out, no cell-local fallback on real rows. True fan-out (a row with several non-null keys) has no viable pilot; it stays covered by the Phase 3a specs until Phase 4 reaches notes.

Migration details

Each migration is a single CREATE OR REPLACE TRIGGER statement per direction, taking SHARE ROW EXCLUSIVE. The table is never left without a trigger, and the down restores the exact pre-MR trigger (never DROP TRIGGER, which would take ACCESS EXCLUSIVE). The down bodies double as the emergency rollback SQL, executable from a console in minutes with no data-loss window.

LFK children affected if records were lost (orphan check by anti-join): project_repository_states, duo_workflows_workloads, p_ci_workload_variable_inclusions, deployment_clusters.

Rollout constraints

  • Merge only after the feature flag removal (target of this MR): the read path must be unconditional before any trigger is rewritten, otherwise routed records are never consumed on instances with the flag off.
  • Rollback order in production: revert triggers first (SQL above), let the worker drain the sharded tables. Pre-existing note: partition DROP/DETACH does not fire DELETE triggers; unchanged by this MR.

How to verify

  • 107 LFK examples pass (helpers, flow, record store, YAML consistency, cleanup worker), RuboCop clean.
  • Rollback tested with scripts/regenerate-schema -r.
  • psql smoke tests on the real tables: group cluster routed to loose_foreign_keys_namespace_deleted_records.namespace_id, organization cluster to the organization table, p_ci_workloads recorded with the parent table name in the ci database, zero cell-local records in all cases.

References

Edited by Leonardo da Rosa

Merge request reports

Loading
Loading