Skip to content
GitLab
Next
Closed
Issue created by Sergii Pechenizkyi@plastiv

`only/except: merge_requests` for merge request pipelines

Problem to Solve

Currently, it is possible to specify regrexp and special keywords to filter the build branches at .gitlab-ci.yml, but it is not possible to set up a job only to be run on a pipeline associated with a merge request. This would be useful for defining different behavior for merge requests - for example additional tests, or other deployment behaviors that are validated in the context of a potential merge. Alternatively, it would be useful to relax day-to-day branches checks without compromising full automation checks for merge requests. I noticed that will full-checks enabled for every branch developers are discouraged to upload their half-baked branches on server due to the flood of failing ci messages.

This item needs to lead into https://gitlab.com/gitlab-org/gitlab-ee/issues/7380; behaviors should be the same and it should not solve a separate, unrelated problem.

Solution

We will add merge_requests as a keyword for only/except. This is the main thrust of this MVC issue, and allows us to start having a kind of pipeline that runs with context of a merge request. This was chosen as the first step because it is the simplest scenario and we don't have to solve all the problems at first. We will still always trigger a non-MR pipeline, just as we do today.

job:
  except:
    - merge_requests

For a pipeline running in the context of a merge request, we expose the merge request ID and target branches as variables.

UI

  1. MR page - Pipeline tab

  2. Pipeline index page

  • Adding a tag which represents that this is a "merge request pipeline" (a pipeline run in the merge request context)
  • Tooltip: This pipeline is run in a merge request context

image

  1. Pipeline detail page

image

Behavior

There should just be a few possible cases here, which are evaluated at git push time (as it works today) and when a new MR is created (this is a new behavior). Important note: An overrriding, pre-existing rule here is that if any pipeline resolves to having no jobs due to only/except rules, that pipeline is not started.

In all cases, a regular pipeline (not in context of the merge request) is attempted to be started. To determine if an additional pipeline in the context of the MR is attempted, the following rules are checked:

  • gitlab-ci.yml contains an only: merge_requests and/or except: merge_requests clause in one or more jobs, and there is a relevant MR available

For this scenario, the behavior is to attempt to trigger an additional merge request pipeline that runs in the context of the merge request. The reason a pipeline would be created in this case is because behaviors are being defined that depend on if it is an MR pipeline.

  • gitlab-ci.yml contains neither only: merge_requests nor except: merge_requests, though there is a relevant MR available
  • gitlab-ci.yml contains an only: merge_requests and/or except: merge_requests clause in one or more jobs, but no relevant MR is created
  • gitlab-ci.yml contains neither only: merge_requests nor except: merge_requests clause in one or more jobs, and no relevant MR is created

For all these scenarios, no attempt to trigger an additional merge request pipeline is made. No behaviors were defined that are relevant to merge requests, and/or no merge request context was available, so there is nothing special to do here.

Upon pipeline completion, all pipelines related to a merge request appear in the pipelines list, but merge request pipelines are preferentially shown in the merge request widget if there are two pipelines for the latest SHA.

Handling different per-branch behaviors

Because the behavior of the pipeline can be different for the source branch vs. the target branch (based on implementation of only/except rules), it's important that users of this feature keep behavior consistent. If they are not consistent, we are not able to fully guarantee that the merged branch will work. For example: if your source branch performs a smaller subset of tests for speed reasons, then running the merged code does not actually validate the same behavior in the target branch since the target branch runs a larger, more comprehensive set of tests that we can not guarantee will pass.

New predefined variables

New predefined variables will be added with the following names.

  • CI_MR_IID ... Merge request IID
  • CI_MR_TARGET_BRANCH ... Merge request target branch

If it's not an MR pipeline, those variables will be empty.

Forking Workflows

We've left forking out of scope intentionally for now, but we will revisit once we have other MR workflows working and can revisit the more complex permissions model that would be required for supporting builds of forks.

This means,

  • When a user pushed a new change to an MR in a parent project, an MR pipeline will be created in the parent project
  • When a user pushed a new change to an MR in a forked project, an MR pipeline will be created in the forked project

This does not affect the current security-level that forked project cannot steal secret variables, specific runners, etc from the parent project.

The permission model is a subject to change in https://gitlab.com/gitlab-org/gitlab-ce/issues/23902

Backend TODO

Frontend TODO

  • Show MR pipeline in MR widget if available
  • Show merge request tag in a pipeline row if it's MR pipeline

Next steps

Auto-canceling the non-MR pipeline if it's running for the same SHA when the MR one is created is something some people may want, and will be looked at as part of https://gitlab.com/gitlab-org/gitlab-ce/issues/32741.

This MR represents a first step towards issues like https://gitlab.com/gitlab-org/gitlab-ce/issues/23902 and https://gitlab.com/gitlab-org/gitlab-ee/issues/7380, and ultimately release trains.

Documentation

We should update https://docs.gitlab.com/ee/ci/yaml/#only-and-except-simplified with the new merge_requests option, being sure to explain the behavior outlined above in detail and why you would use this.fork

Related

Edited by Jason Yavorska