Flexible rules for Pipeline Jobs
Problem to solve
Today we use
only/except to define when to run jobs. This is hard to understand due to the multi-context based approach. Consider the following example, and try to decipher when it is going to be run?
only: refs: - master - /^-stable/ - tags - schedules - merge-requests variables: $CI_COMMIT_TITLE =~ /run me/` changes: - Dockerfile except: refs: - /^v1.*/ changes: - docs/
The rigid structure of the existing
only/except logic coupled with the extension to simple vs. complex logic can cause it to be very hard to decide (as a human) when we expect a particular job to be run. Also, it also limits the ability to create what is a seemingly simple conjunction. (If this or that for instance). It also limits or makes it impossible to define when to run job flexibly.
This problem is orthogonal to:
only/except, but do not extend it. Provide a flexible evaluation of ruleset replacing
Prefer to use a newly introduced two-level
on: is a list of conditions that are traversed,
we would accept or reject based on first matching condition.
All items of the condition have to be matching to be accepted
or rejected. The default policy is
We define a
global: on: that is always evaluated first,
and we define job-specific
on: that is evaluated next.
The global and local has to be matching, so global works as a general filter, pipeline-wide, when local allows you to fine tune matching.
The syntax for both
on: is the same and is represented by the following definition:
on: - branch: my-branch-name project: name-of-my-project # optional - branch: /regexp-for-my-branch-name/ project: name-of-my-project # optional - tag: my-tag-name project: name-of-my-project # optional - tag: /regexp-for-my-tag-name/ project: name-of-my-project # optional - merge_request: name-of-source-branch branch: /name-of-target-branch/ # optional project: name-of-target-project # optional - branch: master source: schedule # push, etc. - branch: master variables: '($CI_COMMIT_TITLE =~ /run me/ || $CI_COMMIT_TITLE =~ /run2 me/) && $MY_VARIABLE == 1' # maybe variables: should be named `if:` to document a custom condition - branch: master triggered_by: project: my-other-project - branch: master protected: true # run when branch or tag is protected - branch: /stable-ee/ policy: accept - branch: /stable/ policy: reject
All the params are optional, each param define a single condition in AND for one Rule, you are free to pick ones interesting.
global: on: ^ above on docker-build: on: - changes: Dockerfile/ docs: on: - changes: docs/
We evaluate a conditions in order, we pick the first one matching and execute the desired
We evaluate the first
global:on: and then
job:on, both has to be matching. When we find first matching this takes an effect.
on: nil it means that it results to be evaluate
true, so it is always matching.
This is based on assumption that current
only/except is a mess, not flexible mess, that is multi-context based,
and does not allow you to flexibly define when to run jobs and does not really allow you to define a specific behavior
The above proposal changes the problem to different definition, makes it flat, easy to understand on first glance, and descriptive based on context. The idea here is that since you see a list of conditions, and you see a policy you will quickly be able to understand how this rule is defined.
I don't think that it is possible to fix
only/except due to it's broken and multi-context nature. Trying to fix it
will make it even harder to understand, and more complex to test. Introducing new keyword
on: allows us to
ensure that you can only use
only/except and vice-versa.