Improve migration squashing to remove orphaned background migration files

What does this MR do?

Enhances the migration squashing rake task to automatically clean up orphaned background migration files that are left behind when old migrations are deleted.

Changes

lib/gitlab/database/migrations/squasher.rb:

  • Added background_migration_files_to_delete method to identify orphaned background migration files
  • Added find_finalized_background_migrations to check YAML files in db/docs/batched_background_migrations/ and find migrations finalized by deleted migrations
  • Added class_name_to_file_name helper to convert CamelCase class names to snake_case file names

lib/tasks/gitlab/db/migration_squash.rake:

  • Integrated background migration file cleanup into the squashing workflow
  • Added output to show how many background migration files are being deleted

spec/lib/gitlab/database/migrations/squasher_spec.rb:

  • Added comprehensive specs for the new background_migration_files_to_delete method
  • Tests YAML file parsing and finalized_by timestamp matching
  • Tests conversion of class names to file paths
  • Tests edge cases (no finalized_by field, finalize migration not deleted)

How it works

  1. When migrations are squashed, the tool collects all deleted migration timestamps
  2. It scans all YAML files in db/docs/batched_background_migrations/
  3. For each YAML file, it checks if the finalized_by timestamp matches a deleted migration
  4. If matched, it extracts the migration_job_name (e.g., BackfillWidgetsNamespaceId)
  5. It converts the class name to snake_case (e.g., backfill_widgets_namespace_id)
  6. It identifies three files to delete:
    • Implementation: lib/gitlab/background_migration/backfill_widgets_namespace_id.rb
    • Spec: spec/lib/gitlab/background_migration/backfill_widgets_namespace_id_spec.rb
    • Documentation: db/docs/batched_background_migrations/backfill_widgets_namespace_id.yml

This approach correctly handles cases where a background migration is referenced by multiple migrations (queue, finalize, cleanup, etc.) - it only removes the files when the finalize migration is deleted.

Related to

#213858 (closed)

Edited by Heinrich Lee Yu

Merge request reports

Loading