fix(rules): Evaluate a job's rules against its own scope

Paweł Farysrequested to merge
gitlab-community/gitlab-org/ci-cd/runner-tools/glci:glci-157 into main

What this MR does and why?

rules: were evaluated against the pipeline-wide variable map alone. A job's own variables: block and the values parallel: matrix: injects were not in that map, so any rule testing them read an empty string and the job was silently dropped. GitLab creates those jobs.

The sharp edge was that glci's own diagnostic agreed with the wrong answer. glci variables printed the rule trace as $SKIP () == "false" → false and, two lines below, a table reporting SKIP = false from source job. That inspector was asked for in #87 (closed), shipped in !123 (merged), and the docs call it "the main way to debug why a rule did or did not match" — on this bug it pointed the wrong way.

A job's rules now see the job's own scope, layered over the pipeline-wide set. Matrix expansion already happened at parse time, so each shard is judged against its own values, which is what makes per-shard rules: work.

The job layer sits where GitLab puts it — below everything a person supplied for the run. Gitlab::Ci::Variables::Builder#scoped_variables_for_pipeline_seed concatenates a job's yaml_variables before user_defined_variables, and the last value for a key wins. So instance, group and project CI/CD variables decide a rule the job also declares, as do the pipeline variables that --env, --env-file, a preset's env and .glci.env stand in for. Global variables the parser folds into every job keep global precedence rather than being promoted, so --env still decides a rule the job merely inherits.

All four rule surfaces take the layer from one helper. The planner filter, the needs: back-fill, the daemon's --show-variables trace and glci variables each built their own view before; that is how the job layer came to be missing from some and not others. The two planner passes also held byte-identical copies of the evaluate-and-clone block, now shared.

The trace is snapshotted before .glciconfig.toml overrides run, and read from the pre-rule job rather than the planner's clone — otherwise the diagnostic could report a rule matching on a variable that rule itself had set.

Out of scope, filed separately: CI_NODE_INDEX/CI_NODE_TOTAL and the rest of the per-job predefined set are still invisible to rules:. They belong below a job's own variables:, which needs a layer this MR does not add. Behaviour there is unchanged.

Steps to reproduce

variables:
  GLOBALVAR: "g"

sees_global:
  image: alpine:3.20
  script: echo "GLOBALVAR=[$GLOBALVAR]"
  rules:
    - if: '$GLOBALVAR == "g"'

sees_job_var:
  image: alpine:3.20
  variables:
    JOBVAR: "j"
  script: echo "JOBVAR=[$JOBVAR]"
  rules:
    - if: '$JOBVAR == "j"'

job_var_overrides_global:
  image: alpine:3.20
  variables:
    GLOBALVAR: "overridden"
  script: echo "GLOBALVAR=[$GLOBALVAR]"
  rules:
    - if: '$GLOBALVAR == "overridden"'

sees_matrix_var:
  image: alpine:3.20
  parallel:
    matrix:
      - TARGET: [a, b]
  script: echo "TARGET=[$TARGET]"
  rules:
    - if: '$TARGET == "a"'

glci jobs

Before — one job of the four GitLab creates:

JOB          STAGE  WHEN        ALLOW_FAILURE  NEEDS
sees_global  test   on_success

After — GitLab's set exactly, with the matrix job as a single shard:

JOB                       STAGE  WHEN        ALLOW_FAILURE  NEEDS
job_var_overrides_global  test   on_success
sees_global               test   on_success
sees_job_var              test   on_success
sees_matrix_var: [a]      test   on_success

glci variables stops contradicting itself. Before, for the same job in the same report:

  Rules:
    #0  $JOBVAR () == "j"  → false

  Variables:
    KEY     VALUE  SOURCE
    JOBVAR  j      job

After:

  Rules:
    #0  $JOBVAR (j) == "j"  → true

Precedence is unchanged where a higher layer defines the key — glci jobs --env GLOBALVAR=other still drops sees_global.

Verified against real GitLab with the Lint API (dry_run=true&include_jobs=true) on a private project: GitLab creates all four, the matrix one as [a] only.

Relevant issues and other links

Closes #157 (closed)

Merge request reports