CI Variables for Merge Trains

Problem to solve

Merge Train Pipelines cannot be differentiated from Merge Request Pipelines via CI variables.

Intended users

The User Persona Devon (DevOps Engineer) will benefit from this feature. A proper solution will be useful in optimizing CI pipelines and runner usage.

Further details

There are cases where you may want to parameterize CI jobs around the Pipeline context.

Examples:

• In a Merge Request Pipeline, you introduce a start stage with a single, manually triggered CI job. The start CI job is started by developers when their code is ready for testing. In this scenario, a Merge Train Pipeline should omit the start CI job because developers have implicitly opted into testing.
• In a Merge Request Pipeline, all CI jobs create a test suite of unit and integration tests. This test suite provides enough validation during development and executes quickly enough for rapid iterations. In a Merge Train Pipeline, the test suite expands to include more integration tests and other time-consuming validation tests. In combination, full test coverage is achieved before merging to master with the advantage of optimized Pipelines during development.

Proposal

Proposed solutions:

• Add a predefined CI variable CI_IS_MERGE_TRAIN which is set to true, otherwise it is not defined at all.
• Create a merge_train job ref that is a superset of merge_request.

In the example of a manual start to pipelines, let's define a sample CI job:

start-pipeline:
    stage: start
    allow_failure: false
    when: manual
    only:
        refs:
            - merge_requests
    script:
        - echo 'Starting pipeline'

With the introduction of a CI variable, the start-pipeline job can be omitted in Merge Trains with the following definition:

start-pipeline:
    stage: start
    allow_failure: false
    when: manual
    only:
        refs:
            - merge_requests
    except:
        variables:
            - $CI_IS_MERGE_TRAIN
    script:
        - echo 'Starting pipeline'

With the introduction of a job ref, the start-pipeline job can be omitted in Merge Trains with the following definition:

start-pipeline:
    stage: start
    allow_failure: false
    when: manual
    only:
        refs:
            - merge_requests
    except:
        refs:
            - merge_trains
    script:
        - echo 'Starting pipeline'

Permissions and Security

There are no changes to permissions in this case.

Documentation

The Gitlab CI/CD documentation will need to be updated.

Testing

The risk depend on the final solution.

With a new CI variable, test coverage should assert environment variable definition and value validation in the relevant scenarios.

With a new job ref, there is a greater risk to pipeline validation. Proper relationships must be maintained between merge_request and merge_train job refs.

What does success look like, and how can we measure that?

A successful solution to this problem will allow a Developer to write a CI job that is conditionally executed in either/both Merge Train and Merge Request pipelines through CI configuration alone.

What is the type of buyer?

This feature is relevant to customers with Merge Trains, which is only available to Premium and Silver tiers or higher.

Links / references