Skip to content

Allow variable substitution in .gitlab-ci.yml only/except "variables" key

Problem to solve

Say I have a CI config with multiple jobs (in my real use case I might have dozens, which motivates the feature request, but it's illustrated here with just a couple jobs for brevity):

job1:
    script: job1.sh

job2:
    script: job2.sh

It would be nice if the pipeline had a variable that I could pass in that would explicitly let me filter what jobs I execute (for instance, if I have a pipeline with many jobs in it, but I only want to run one of them to test something, then this could save on resources).

Proposal

This could look something like:

job1:
    script: job1.sh
    only:
        variables:
            - $JOB_PATTERN == null
            - $CI_JOB_NAME ~= /$JOB_PATTERN/

job2:
    script: job2.sh
    only:
        variables:
            - $JOB_PATTERN == null
            - $CI_JOB_NAME ~= /$JOB_PATTERN/

So then when the pipeline is created, it will do the following:

  • If the JOB_PATTERN variable is not set, it will run both jobs.
  • Otherwise:
    • If job1 matches the regex contained by JOB_PATTERN, then it will be run.
    • If job2 matches the regex contained by JOB_PATTERN, then it will be run.

However, this would require variable substitution inside the regex pattern, which is probably tricky since the $ has a specific meaning for the regex. There could be a better way to do this, including integrating it more deeply into the only/except process instead of hooking the existing variable expression form.

What does success look like, and how can we measure that?

I would call it success if, in the Web UI, I could easily access the Run Pipeline functionality and create a new pipeline with a filter that limits exactly which of the jobs in the configuration will be run.