Extend 'only' and 'except' to add environment variables
Description
This is the original intent of #34785 (closed) that was then addressed differently, but the initial idea is valuable and allows to do more powerful improvements to Auto DevOps template (and also others).
Extending syntax, we add capabilities to select if a job should be executed or not if specific environment variables are set, extending only
and except
. This is good since it allows to avoid creating the job if conditions are not met, saving a lot of runner resources.
Proposal
Extend only the complex scope of only
and except
, using the new keyword variables
that is in logical AND as it is now for refs
and kubernetes
.
Note: This syntax change has to be considered alpha, and subject to change.
This is the proposed syntax:
only:
variables:
- $VAR1 # variable present, defined and non-empty
- $VAR2 == "yes" # variable content equals to "yes"
- $VAR3 == $VAR4 # variable on the left equals to variable on the right
- $VAR5 == "" # variable defined but empty
Notes:
- items on different lines are in logical OR
- string values must be enclosed between
""
- it is not possible to do variable interpolation in strings
- it is not possible to escape chars in strings
- no support for conjunction yet (in the first iteration)
Old version:
job:
only:
refs:
- master
- schedules
- branches
- tags
files:
- ./Dockerfile
expressions:
- $KUBECONFIG && $KUBE_DOMAIN
This job will be scheduled only if refs
match one of the listed items, AND a file named ./Dockerfile
exists, AND if both variables KUBECONFIG
and KUBE_DOMAIN
are set.
Keywords in only
and except
are considered in logical AND, but keys in refs
, files
, expressions
are considered in logical OR. Syntax for expressions
allows &&
. If logical OR is needed, you can just use two different list items.
Links / references
- Auto DevOps (#35712 (closed))
- Original discussion (#34785 (closed))
Feature checklist
Make sure these are completed before closing the issue, with a link to the relevant commit.
-
Feature assurance -
Documentation -
Added to features.yml
Related issues:
Proposal from https://gitlab.com/gitlab-org/gitlab-ce/issues/23538
The only and except parameters in .gitlab-ci.yml are very useful for limiting runs. I have a system where each build is an arbitrary subset of the possible large superset of builds (these are recipes to build other software - I only want to build any recipes in my repository that have changed). I can submit individual builds and use an environment variable to carry out the selected build. However, I can't limit the tags/platforms that it runs on. I am left with something of a hack: run all jobs for all platforms all of the time, but terminate non-matching platforms early based on the trigger variables. This still costs me machine spin-up time. If I bundle all platforms into one build, then the dependency build-out is unnecessarily constrained by waiting for all of the platforms to finish a given dependency - even if they don't use it.
I would like to propose that environment variable expressions be usable in the only/except parameter, so that they can be used to limit which jobs actually run on triggered builds.
Example:
some_job: only: - $TARGET_PLATFORM == some_hardcoded_tag
or, even cooler, relate to tags (check for membership?):
some_job: only_in_tags: - $TARGET_PLATFORM
Alternatively, allowing trigger variables in the tags would be a good alternate to the functionality I'd like.
some_job: tags: - $TARGET_PLATFORM