CI: Skippable steps should not block "merge when pipeline succeeds"
Problem to solve
We use the pipeline to not only test but also to generate reports and metadata useful for testing again in the future.
We should be able to speed up a "Merge when pipeline succeeds" by considering a pipeline done for that purpose when any non allow_failure
job is green.
Further details
By using either the allow_failure
as a flag to conder pipeline green, or another new flag specifically designed for that, we speedup merge operations, delaying to after it was merged the generation of "javascript reports", or other jobs that are not essential.
Proposal
Use either allow_failure
or create a new stage for the non-essential jobs (the ones that are either reports or notifications etc) so we can get merge request merged before processing those, recovering back precious minutes. Consider when a merge request is a dependency for another one, if we can shave-off few minutes from each, that means they both go to master faster
What does success look like, and how can we measure that?
Be able to have "something" that allows us to have a test being running normally during Merge Request lifecycle and that would not block "merge with pipeline succeeds", and will allow it to execute even before running them.