Skip to content

Extend 'only' and 'except' to add environment variables

Description

This is the original intent of #34785 (closed) that was then addressed differently, but the initial idea is valuable and allows to do more powerful improvements to Auto DevOps template (and also others).

Extending syntax, we add capabilities to select if a job should be executed or not if specific environment variables are set, extending only and except. This is good since it allows to avoid creating the job if conditions are not met, saving a lot of runner resources.

Proposal

Extend only the complex scope of only and except, using the new keyword variables that is in logical AND as it is now for refs and kubernetes.

Note: This syntax change has to be considered alpha, and subject to change.

This is the proposed syntax:

only:
  variables:
    - $VAR1           # variable present, defined and non-empty
    - $VAR2 == "yes"  # variable content equals to "yes"
    - $VAR3 == $VAR4  # variable on the left equals to variable on the right
    - $VAR5 == ""     # variable defined but empty

Notes:

  • items on different lines are in logical OR
  • string values must be enclosed between ""
  • it is not possible to do variable interpolation in strings
  • it is not possible to escape chars in strings
  • no support for conjunction yet (in the first iteration)

Old version:

job:
  only:
    refs:
      - master
      - schedules
      - branches
      - tags
    files:
      - ./Dockerfile
   expressions:
    - $KUBECONFIG && $KUBE_DOMAIN

This job will be scheduled only if refs match one of the listed items, AND a file named ./Dockerfile exists, AND if both variables KUBECONFIG and KUBE_DOMAIN are set.

Keywords in only and except are considered in logical AND, but keys in refs, files, expressions are considered in logical OR. Syntax for expressions allows &&. If logical OR is needed, you can just use two different list items.

Links / references

Feature checklist

Make sure these are completed before closing the issue, with a link to the relevant commit.

Related issues:

Proposal from https://gitlab.com/gitlab-org/gitlab-ce/issues/23538

The only and except parameters in .gitlab-ci.yml are very useful for limiting runs. I have a system where each build is an arbitrary subset of the possible large superset of builds (these are recipes to build other software - I only want to build any recipes in my repository that have changed). I can submit individual builds and use an environment variable to carry out the selected build. However, I can't limit the tags/platforms that it runs on. I am left with something of a hack: run all jobs for all platforms all of the time, but terminate non-matching platforms early based on the trigger variables. This still costs me machine spin-up time. If I bundle all platforms into one build, then the dependency build-out is unnecessarily constrained by waiting for all of the platforms to finish a given dependency - even if they don't use it.

I would like to propose that environment variable expressions be usable in the only/except parameter, so that they can be used to limit which jobs actually run on triggered builds.

Example:

some_job: only: - $TARGET_PLATFORM == some_hardcoded_tag

or, even cooler, relate to tags (check for membership?):

some_job: only_in_tags: - $TARGET_PLATFORM

Alternatively, allowing trigger variables in the tags would be a good alternate to the functionality I'd like.

some_job: tags: - $TARGET_PLATFORM

Edited by Fabio Busatto