Skip job if already run on same code
When merging a MR, a CI/CD pipeline is kicked off again. This can be very useful if the target branch has changed since the MR was created and you want to be sure no problems are introduced. But if you have
Fast-forward merge or
Merge commit with semi-linear history enabled, then you know that such changes are not possible and a lot of the CI jobs are a waste.
We should detect such situations and not run redundant jobs. Note that not all jobs can be skipped, specifically deployments are often done only for the
master branch, but not on topic branches, but also, sometimes teams do more extensive tests on
master that aren't done on topic branches. So, you can't skip the entire pipeline, but you could skip individual jobs. Since each job name is unique, an existing successful job of the same name should be sufficient to skip a new run.
Fast-forward merge, you can check for a matching SHA, but for
Merge commit with semi-linear history, there's still a new commit for the merge, so we'd have to be a bit craftier. We could skip that for a first iteration, but if we detect the merge commit well, then we could apply this even for regular merge commits whenever the MR happens to be based off of HEAD of target (because nothing else has been merged since, or because it's been rebased recently).
Also, we could apply the same logic to tags. i.e. if a commit is tagged and pushed, it might trigger new CD actions, but could skip redundant CI tests.
It would be really interesting if you also considered forks or mirrors, where someone catches up to the upstream project and doesn't re-test things that have already been tested. This is likely to have all sorts of edge cases making it impossible or useless, but something interesting to think about.
Links / references
When merging to master, we usually re-run all tests to make sure that the changes in the branch still work after the merge. But if the merge request was already up to date with master, then there's no value in re-running tests if they've already been run. We can't just skip all CI, since the deploy jobs will likely be different. Also, we might have deeper integration tests that we want to run on
master that we wouldn't run on feature branches. But for any job that is exactly the same, we should be able to skip it. I envision something like a
when clause that lets us specify which jobs shouldn't be run if the same job has been run on the exact same git SHA (or that only differs by the merge commit). We'd likely have to track which jobs have succeeded for each git SHA to be certain. We wouldn't want to just skip all jobs for
master, we'd need to be sure the tests actually passed before. Sure, you shouldn't merge unless the tests have already passed, but again, those tests may have passed based on something earlier than HEAD of
master, in which case I'd want to the tests to re-run.