Draft: EvaluateWorkflowRules configuration for branch and merge_request
What does this MR do?
This branch demonstrates a proposed configuration option for controlling branch and pipeline execution in workflow:rules
Currently, the use of always
and never
is not granular enough to let authors configure pipelines to run only for branches or merge requests. So I propose we add branch
and merge_request
.
Using this configuration, we should be able to remove the default job:only
configuration and fully rely on workflow
, which will have explicit configuration:
workflow:
rules:
- if: $CI_OPEN_MERGE_REQUESTS && $RUN_BOTH_PIPES
when: always
- if: $CI_OPEN_MERGE_REQUESTS
when: merge_requests
- if: $CI_PIPELINE_SOURCE == "push"
when: branches
job1:
script: echo "Every pipeline"
job2:
script: echo "not for regular development, only run this when it's part of an open merge request"
rules:
- if: $CI_MERGE_REQUEST_ID
# only:
# - merge_requests
job3:
script : echo "an abbreviated triage job that only runs on development branches that aren't in merge requests"
rules:
- if: $CI_MERGE_REQUEST_ID
when: never
# only:
# - branches
In this example, job1
will be run in every pipeline, branch or merge request, but workflow:rules
prevents two pipelines from ever being created at the same time unless explicitly asked for via $RUN_BOTH_PIPES
. If that rule were to be removed, two pipelines would never be created simultaneously.
- If Delaney pushes a new feature and there is no open merge request, a single pipeline will run with
job1
andjob3
. - Delaney opens a merge request from the new feature branch, a single pipeline will run with
job1
andjob2
. - After pushing to an open merge request, Delaney can go to the web UI and run a new pipeline with custom variable
$RUN_BOTH_PIPES
set totrue
, and will get two new pipelines identical to the ones produced by the push and new merge request.
I think this will be much more obvious to configuration writers in the future.
Configuration outcomes
Using this configuration, we would no longer need the secret default only:[branches,tags]
, because the evaluation of workflow:rules will now stop the processing of merge request pipelines entirely if the workflow rules do not explicitly call for them.
The lack of default job inclusion configuration means that jobs will be included for any pipeline that EvaluateWorkflowRules
says is configured. If no jobs are configured for a pipeline, (e.g. every job has only:[branches]
that pipeline (the MR pipeline with no jobs) will not be created, as is the case today.
I used "only" to describe configurations here because it's easier to read quickly. The sample configuration above has comments to show equivalent configurations.
Workflow Rules when:
|
job1 only:
|
job2 only:
|
Branch Pipeline | MR Pipeline |
---|---|---|---|---|
always |
branches |
--- | job1, job2 | job2 |
always |
branches |
merge_requests |
job1 | job2 |
always |
--- | merge_requests |
job1 | job1, job2 |
always |
--- | --- | job1, job2 | job1, job2 |
branches |
branches |
--- | job1, job2 | --- |
branches |
branches |
merge_requests |
job1 | --- |
branches |
--- | merge_requests |
job1 | --- |
branches |
--- | --- | job1, job2 | --- |
merge_requests |
branches |
--- | --- | job2 |
merge_requests |
branches |
merge_requests |
--- | job2 |
merge_requests |
--- | merge_requests |
--- | job1, job2 |
merge_requests |
--- | --- | --- | job1, job2 |
Some example common configuration
"I have two jobs. I want branch pipelines most of the time, but if I've opened the merge request, run for the merged result and skip the plain branch."
workflow:
rules:
- if: $CI_OPEN_MERGE_REQUESTS
when: merge_requests
- when: branches
# Notice that since the jobs always run in both pipelines, they don't need to be configured!
job1:
script: echo "First job"
job2:
script: echo "Second job"
"I'm adding a third job. It's expensive, so only run it in my merge request pipelines after the other jobs succeed".
costly_job:
script: echo "Really expensive tests"
rules:
- if: $CI_MERGE_REQUEST_ID
needs:
- job1
- job2
Related Issues
#327352 (closed) - By not understanding the default only:
implementation, this pipeline author doesn't understand why the first job is being left out of pipelines. If pipelines were configured explicitly in workflow:
instead of implicitly through job:only
, I think there would be less confusion.
#322524 - An author is unintentionally creating merge request pipelines by writing rules related to CI_PROJECT_NAME
, or other information completely unrelated to merge requests.
#297372 (closed) - An author who isn't getting what they expect out of the Workflow Merge Request Pipelines template. While the template is concise and effective for a particular behavior, I think it would be easier to understand if pipeline author had written explicit configuration asking for branch and/or merge request pipelines.
Adoption
Adoption of this configuration should be fairly intuitive, because always
will still mean always. If people have duplicate pipelines, we can tell them "Well don't say always, just say branches!"
What about tags?
Tags would work exactly the same way, but I have left them out of this for the sake of making it easier to read.
Does this MR meet the acceptance criteria?
Certainly not. Stop reading now, I'll fill out the rest of this if and when it's a real MR.
Conformity
-
📋 Does this MR need a changelog?-
I have included a changelog entry. -
I have not included a changelog entry because _____.
-
-
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers -
Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team