Move the detection of pending post migrations to the main stage.
The execution post-deploy migration (PDM) pipeline relies on pending post migrations, if there are, the pipeline is executed otherwise the execution is skipped.
Pending post migrations are detected in the canary stage, after the regular migrations are executed (through the gprd-cny-fetch-post-migrations
). This could lead to an empty execution of the PDM, that is, triggering the pipeline but not executing any pending post migration (example), this is because the PDM thinks there are pending post migrations to be executed but the post migrations are only on the canary stage and not in the main one.
Proposal
Let's keep the detection of pending post migrations and the execution of the PDM in sync by:
-
Detecting the pending post migrations in the main stage by moving the gprd-cny-fetch-post-migrations
to be executed in gprd after thegprd-migrations
job. -
Make the detection of the pending post migrations to rely on the thanos metrics on main. Broadly the logic should: - Check if there pending post migrations on gprd (
{stage="main"} > 0
) - If there are, fetch the pending post migrations sent by deployer and proceed with the execution
- If there are not, the execution is skipped.
- Check if there pending post migrations on gprd (
Edited by Matt Field