Add orphaned-parent test case to desired sharding key backfill shared examples

Chen Zhangrequested to merge
duo/feature/606942-orphaned-parent-backfill-shared-example into master

What does this MR do and why?

Adds an orphaned-parent test case to the 'desired sharding key backfill job' shared example so every migration using BackfillDesiredShardingKeyJob makes a deliberate, tested choice about what happens when a parent row's backfill_via_column references a deleted project/namespace/organization.

Background

Loose foreign key (LFK) cleanup is eventually consistent. Parent rows whose backfill_via_column references a deleted project/namespace/organization are normal in production, not edge cases. The Sev1 incident (professional_teal_sparrow) occurred because BackfillPackagesHelmMetadataCacheStatesProjectId copied a dangling project_id from an orphaned parent into a child column protected by a hard FK, aborting the upgrade. The fix pattern is in !246021 (merged) (delete orphaned parents per sub-batch, then backfill).

Changes

Shared example (spec/support/shared_examples/lib/gitlab/background_migration/backfill_desired_sharding_key_shared_examples.rb):

Adds a new context 'when a parent row has an orphaned backfill_via_column value' that:

  • Finds a parent row with at least one child in batch_table
  • Corrupts the parent's backfill_via_column to non_existing_record_id via raw SQL (using SET LOCAL session_replication_role = replica to bypass constraints)
  • Sets the child's backfill_column to NULL (simulating pre-backfill state)
  • Runs migration.perform and asserts based on expected_orphan_behavior
  • Skips gracefully if no parent rows with children exist

Supported expected_orphan_behavior values:

  • :copies_dangling_value — default; the unmodified template copies the dangling ID into the child column (safe only when the child column has no hard FK)
  • :deletes_parent — the migration deletes the orphaned parent before backfilling (MR !246021 (merged) pattern)
  • :skips — the child's backfill column stays NULL
  • :raises — the migration raises an error (e.g. FK violation)

Host spec declarations (11 specs updated):

  • :deletes_parent for BackfillPackagesHelmMetadataCacheStatesProjectId
  • :raises for specs where the child column has a hard FK to projects, users, or organizations
  • Default :copies_dangling_value for p_ci_pipeline_artifact_states and ci_build_needs (no hard FK on backfill column)

Documentation (doc/development/organization/sharding/_index.md): Adds a new subsection "Handling orphaned parents in backfill specs" explaining the convention and when to use each behavior.

References

Screenshots or screen recordings

N/A — spec and documentation changes only.

How to set up and validate locally

bundle exec rspec spec/lib/gitlab/background_migration/backfill_packages_helm_metadata_cache_states_project_id_spec.rb
bundle exec rspec spec/lib/gitlab/background_migration/backfill_gpg_key_subkeys_user_id_spec.rb
bundle exec rspec spec/lib/gitlab/background_migration/backfill_deployment_clusters_project_id_spec.rb

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Merge request reports