Conditional downstream branch selection for multi-project pipelines
Problem to solve
As a user using multi-project pipelines, I want to optimise my workflow by automatically selecting a downstream branch for the downstream project when it matches or relates to the branch from upstream.
- I have a project example/component (upstream) which triggers a pipeline in example/project (downstream).
- There may exist a branch in the downstream repo which contains tests to match the work in the upstream repo.
- I'd like to select the downstream branch if it exists as this may include additional tests which relate to the new upstream feature branch.
- Otherwise the default behaviour desired is to trigger the tests from the default downstream branch.
Current functionality & available approaches (as at 2020-07 / Gitlab 13.1.5)
Some of this can be achieved at present by modifying the upstream branch .gitlab-ci.yml
to reflect the branchname downstream, but this introduces additional workflow steps (setting downstream.trigger.branch
, removing downstream.trigger.branch
, and risk of forgetting to remove downstream.trigger.branch
setting when merging upstream).
It's also possible to switch branches conditionally at present as part of the downstream pipeline (my current workaround). This has the limitation that it runs late in the CI pipeline process; the downstream branch is not able to test changes to components which run before the CI script is called, eg changes to .gitlab-ci.yml
in the downstream pipeline.
It might be possible to achieve the same outcome by populating a variable at runtime to the triggering upstream CI job, which is used to set the downstream branch selection? I don't yet see how I'd achieve that.
Intended users
- Delaney (Development Team Lead) can be confident that there is not unnecessary "flapping" with changes to CI configuration.
-
Sasha (Software Developer), Devon (DevOps Engineer), Simone (Software Engineer in Test), Alex (Security Operations Engineer), Priyanka (Platform Engineer) can expect that automated or process-driven branch naming (eg
123-featurename
) will better co-ordinate as they contribute changes to related repositories. - Rachel (Release Manager) can be confident that related changes are managed asynchronously and delivered more rapidly.
User experience goal
The user should be able to use CI with GitLab to develop sets of related changes across projects more easily.
Proposal
I would like to do so "lazily", eg by setting downstream.trigger.branch
conditionally if it already exists downstream, or by having Gitlab use the default downstream branch if no such branch exists.
I know I can implement an approach in the pipeline using a script and variables, but this runs later so there are limitations to that approach.
Further details
Permissions and Security
Input welcome here.
Documentation
Proposed documentation (intended as illustrative at this stage):
Specifying a downstream pipeline branch
It is possible to specify a branch name that a downstream pipeline will use:
rspec:
stage: test
script: bundle exec rspec
staging:
stage: deploy
trigger:
project: my/deployment
branch: stable-11-2
Use:
- The
project
keyword to specify the full path to a downstream project. - The
branch
keyword to specify the name of a branch in the project specified byproject
. From GitLab 12.4, variable expansion is supported. - The
branch_default
keyword to specify a branch to use when the branch selected inbranch
is not available.
GitLab will use a commit that is currently on the HEAD of the branch when creating a downstream pipeline.
Setting Branch and Default Branch
To trigger the downstream pipeline to run with a branch which relates to the upstream branch, you can set branch
to a variable and branch_default
to a sensible default. For example:
rspec:
stage: test
script: bundle exec rspec
staging:
stage: deploy
trigger:
project: my/deployment
branch: ${CI_COMMIT_BRANCH}
branch_default: master
Draft content only, this is just an FR currently :)
Availability & Testing
What does success look like, and how can we measure that?
Easier and less complicated to deliver related changes through multi-project pipelines.