`only/except: merge_requests` for merge request pipelines
Problem to Solve
Currently, it is possible to specify regrexp and special keywords to filter the build branches at
.gitlab-ci.yml, but it is not possible to set up a job only to be run on a pipeline associated with a merge request. This would be useful for defining different behavior for merge requests - for example additional tests, or other deployment behaviors that are validated in the context of a potential merge. Alternatively, it would be useful to relax day-to-day branches checks without compromising full automation checks for merge requests. I noticed that will full-checks enabled for every branch developers are discouraged to upload their half-baked branches on server due to the flood of failing ci messages.
This item needs to lead into https://gitlab.com/gitlab-org/gitlab-ee/issues/7380; behaviors should be the same and it should not solve a separate, unrelated problem.
We will add
merge_requests as a keyword for only/except. This is the main thrust of this MVC issue, and allows us to start having a kind of pipeline that runs with context of a merge request. This was chosen as the first step because it is the simplest scenario and we don't have to solve all the problems at first. We will still always trigger a non-MR pipeline, just as we do today.
job: except: - merge_requests
For a pipeline running in the context of a merge request, we expose the merge request ID and target branches as variables.
MR page - Pipeline tab
Pipeline index page
- Adding a tag which represents that this is a "merge request pipeline" (a pipeline run in the merge request context)
This pipeline is run in a merge request context
- Pipeline detail page
- https://gitlab.com/gitlab-org/gitlab-ce/issues/54336 Making pipeline tags visible (separate issue)
CI_MR_TARGET_BRANCHinside branch specification
There should just be a few possible cases here, which are evaluated at
git push time (as it works today) and when a new MR is created (this is a new behavior). Important note: An overrriding, pre-existing rule here is that if any pipeline resolves to having no jobs due to only/except rules, that pipeline is not started.
In all cases, a regular pipeline (not in context of the merge request) is attempted to be started. To determine if an additional pipeline in the context of the MR is attempted, the following rules are checked:
except: merge_requestsclause in one or more jobs, and there is a relevant MR available
For this scenario, the behavior is to attempt to trigger an additional merge request pipeline that runs in the context of the merge request. The reason a pipeline would be created in this case is because behaviors are being defined that depend on if it is an MR pipeline.
except: merge_requests, though there is a relevant MR available
except: merge_requestsclause in one or more jobs, but no relevant MR is created
except: merge_requestsclause in one or more jobs, and no relevant MR is created
For all these scenarios, no attempt to trigger an additional merge request pipeline is made. No behaviors were defined that are relevant to merge requests, and/or no merge request context was available, so there is nothing special to do here.
Upon pipeline completion, all pipelines related to a merge request appear in the pipelines list, but merge request pipelines are preferentially shown in the merge request widget if there are two pipelines for the latest SHA.
Handling different per-branch behaviors
Because the behavior of the pipeline can be different for the source branch vs. the target branch (based on implementation of only/except rules), it's important that users of this feature keep behavior consistent. If they are not consistent, we are not able to fully guarantee that the merged branch will work. For example: if your source branch performs a smaller subset of tests for speed reasons, then running the merged code does not actually validate the same behavior in the target branch since the target branch runs a larger, more comprehensive set of tests that we can not guarantee will pass.
New predefined variables
New predefined variables will be added with the following names.
CI_MR_IID... Merge request IID
CI_MR_TARGET_BRANCH... Merge request target branch
If it's not an MR pipeline, those variables will be empty.
We've left forking out of scope intentionally for now, but we will revisit once we have other MR workflows working and can revisit the more complex permissions model that would be required for supporting builds of forks.
- When a user pushed a new change to an MR in a parent project, an MR pipeline will be created in the parent project
- When a user pushed a new change to an MR in a forked project, an MR pipeline will be created in the forked project
This does not affect the current security-level that forked project cannot steal secret variables, specific runners, etc from the parent project.
The permission model is a subject to change in https://gitlab.com/gitlab-org/gitlab-ce/issues/23902
only/except: merge_requestskeywords in .gitlab-ci.yml => https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/23217
- Create merge request pipelines when merge request is created/updated => https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/23217
Expose predefined variable
- Expose Expose merge request flag => https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/23399
- Show MR pipeline in MR widget if available
merge requesttag in a pipeline row if it's MR pipeline
Auto-canceling the non-MR pipeline if it's running for the same SHA when the MR one is created is something some people may want, and will be looked at as part of https://gitlab.com/gitlab-org/gitlab-ce/issues/32741.
This MR represents a first step towards issues like https://gitlab.com/gitlab-org/gitlab-ce/issues/23902 and https://gitlab.com/gitlab-org/gitlab-ee/issues/7380, and ultimately release trains.
We should update https://docs.gitlab.com/ee/ci/yaml/#only-and-except-simplified with the new merge_requests option, being sure to explain the behavior outlined above in detail and why you would use this.fork