Improve migration squashing to remove orphaned background migration files
What does this MR do?
Enhances the migration squashing rake task to automatically clean up orphaned background migration files that are left behind when old migrations are deleted.
Changes
lib/gitlab/database/migrations/squasher.rb:
- Added
background_migration_files_to_deletemethod to identify orphaned background migration files - Added
find_finalized_background_migrationsto check YAML files indb/docs/batched_background_migrations/and find migrations finalized by deleted migrations - Added
class_name_to_file_namehelper to convert CamelCase class names to snake_case file names
lib/tasks/gitlab/db/migration_squash.rake:
- Integrated background migration file cleanup into the squashing workflow
- Added output to show how many background migration files are being deleted
spec/lib/gitlab/database/migrations/squasher_spec.rb:
- Added comprehensive specs for the new
background_migration_files_to_deletemethod - Tests YAML file parsing and finalized_by timestamp matching
- Tests conversion of class names to file paths
- Tests edge cases (no finalized_by field, finalize migration not deleted)
How it works
- When migrations are squashed, the tool collects all deleted migration timestamps
- It scans all YAML files in
db/docs/batched_background_migrations/ - For each YAML file, it checks if the
finalized_bytimestamp matches a deleted migration - If matched, it extracts the
migration_job_name(e.g.,BackfillWidgetsNamespaceId) - It converts the class name to snake_case (e.g.,
backfill_widgets_namespace_id) - It identifies three files to delete:
- Implementation:
lib/gitlab/background_migration/backfill_widgets_namespace_id.rb - Spec:
spec/lib/gitlab/background_migration/backfill_widgets_namespace_id_spec.rb - Documentation:
db/docs/batched_background_migrations/backfill_widgets_namespace_id.yml
- Implementation:
This approach correctly handles cases where a background migration is referenced by multiple migrations (queue, finalize, cleanup, etc.) - it only removes the files when the finalize migration is deleted.
Related to
Edited by Heinrich Lee Yu