Require jobs with allow_failure to pass if MR changes the jobs behaviour
Problem:
Given some jobs are allowed to fail in the pipeline, when an MR changes the behaviour of these jobs in a way that would break the job, the pipeline must be able to prevent the MR from being merged.
We need a reliable way to catch failures from changes that would change the behaviour of those allowed to fail
jobs.
Stages where this problem could occur:
-
review
stage -
qa
stage -
prepare
stage -
post-test
stage
Context:
Review app deployment was completely broken due to ~backstage change !30272 (merged). This would have been caught on the MR automatically if the review-deploy
job has not been allowed to fail for this MR.
Proposal:
Disable allow_failure
on specific jobs when an MR change is related to the job behaviour. For example, any change in scripts/review_apps
should require review app related jobs to pass.