Database migration: warn when validating not null constraint on field that was backfilled via background migration
Problem
Related issue: #406885 (comment 1431747409)
I thought that I correctly followed the instructions to add a not null constraint on an existing column:
- %15.8: MR with batched background migration to update existing records
- %15.8: MR to add not null constraint to table (with validate: false )
- %15.10: MR to validate not null constraint.
But now, I can see that as part of %15.10 I should have added a separate post-deployment migration to check that the batched background migration was completed before validating the not-null constraint on the column. These piece that we missed is a bit buried in the batched background migration documentation here:
Release B: 1. Add a post-deployment migration that checks if the batched background migration is completed.
Solutions
Improve documentation
One iteration to improve this in the future is to ensure that we improve the documentation so that we include information about adding checks if related batched background migrations, if there are any, are completed in the documentation on validating not null constraints.
Bot warnings
In the MR where we validated the not null constraint, this issue ideally should have been flagged by a bot.
A naive bot could just add a warning whenever a migration is added that includes validate_not_null_constraint
and suggests checking whether there is a background job being used to backfill the column where the constraint is being validated.
A smarter bot could add a warning only if a background migration is found for that column and ensure_batched_background_migration_is_finished
is not found in any migrations