Streamline merge request pipelines

We run at least 4 3 CI pipelines for the same repository state for every commit and usually several more. I propose we remove 1 ~~or 2~~ of those, and trigger entire pipelines only for a push or merge_request_event (never both).

Note 1: The merge train pipeline duplicates the final merge request pipeline, wasting resources and slowing down the merge process without adding utility.

Note 2: Pipelines triggered by merge requests do not include all of the tests that we consider essential because we divide some jobs between pipelines that we expect to run concurrently.

Background

GitLab CI is architected with the expectation that we run one pipeline per commit, and only the pipeline triggered by the merge request event is considered for "require pipeline success" merge request checks.

GitLab assumes that customers will not be using both "Merge Trains and pipelines for merged results" and "Pipelines must succeed" options at the same time because they are redundant: the purpose of the merge train is to enforce a required successful pipeline before merge.

Merge trains do more harm than good in our fast-forward-only workflow. They do not perform the automatic rebase that is required to maintain multiple pending merges and they merely duplicate the final pipeline of a MR that is ready to be fast-forward squash-merged.

Proposal

Make sure that exactly zero or one pipeline runs for each ref update. Alternatively, make sure the important pipeline is the merge request pipeline: https://gitlab.com/help/user/project/merge_requests/merge_when_pipeline_succeeds.md#limitations See also https://gitlab.com/help/ci/yaml/README.md#differences-between-rules-and-onlyexcept

Suggestion: Use the logic suggested in https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Workflows/MergeRequest-Pipelines.gitlab-ci.yml to disable branch push pipelines in the gromacs project namespace. Make sure that blocking test jobs are included in the pipelines triggered for merge requests.

Additional notes and consequences

It becomes possible to push a branch that does not trigger any pipelines.

Pipelines will no longer be automatically triggered for pushes to https://gitlab.com/gromacs/gromacs branches if those branches are not tied to active merge requests. Pipelines can be manually triggered for any such branch through the https://gitlab.com/gromacs/gromacs/-/pipelines web interface or by pushing to a fork instead of the main repository.

Note that when GitLab 13.3 is released in late August, it is possible to Create pipelines in the parent project for merge requests from a forked project

More jobs will execute in multiple pipelines...

... but fewer pipelines will run. The intention of Proposal 1 is to avoid running pipelines that don't actually block merge requests, while making sure that the pipelines that can block merge requests are complete.

Job rules can be simplified

We currently evaluate the pipeline trigger source for push and merge_request_event types. It is more relevant to apply logic regarding whether the job should run on free resources or whether it requires special GROMACS project resources. We can move the push versus merge_request_event logic to the workflow level (based additionally on source and target branch per conventional workflow:rules, and on whether $CI_PROJECT_NAMESPACE != "gromacs")

Progress

A related proposal was adopted 30 July 2020

Disable "Merge Trains and pipelines for merged results".

The option is redundant with "Pipelines must succeed" and does not provide additional utility to us, because we already require fast-forward-only merges.

Alternate

If proposal 2 is rejected for some reason, we need to make sure that the pipeline that is executed actually provides a useful check. Make sure that "Merge When Pipeline Succeeds" references the correct pipeline. See also gitlab-org/gitlab#12267 (comment 205295612) and https://gitlab.com/help/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md#unable-to-add-to-merge-train-with-message-the-pipeline-for-this-merge-request-failed

Deferred

A pipeline runs for the push event on the master branch after a merge request is merged. If the merge_request_event pipelines had complete coverage, this extra pipeline should not reveal anything new. However, there is some logic in the job scripts that operate on the the git history of HEAD, which may have changed if the merge request was squashed during final submission. If we can or should adjust or eliminate the pipeline triggered for push events on master, let's tackle that in a separate issue.

Edited Nov 18, 2020 by M. Eric Irrgang