Backend: Validation of max includes passes in CI/CD Editor but fails when pipeline runs

A bug was discovered upon implementing the fix (!111391 (closed)) for Backend: Prevent unlimited number of includes f... (#367150 - closed).

There's actually another bug that allows the validation to pass in the CI/CD Editor but fails when you run the pipeline. This happens because when the pipeline runs, the files list additionally includes the .gitlab-ci.yml file, bringing the total file count up [by 1]

How to Reproduce locally

  1. Check out the branch from !111391 (closed)
  2. Update the max includes value to 2 in lib/gitlab/ci/config/external/context.rb:12.
MAX_INCLUDES = 2
  1. Enable the ci_fix_max_includes feature flag.
Feature.enable('ci_fix_max_includes')
  1. Set up a simple config of 2 nested includes by creating the following files in your project root.

File1: template1.yml

job1:
  script: exit 0

File2: template2.yml

include:
  - local: template1.yml
job2:
  script: exit 0
  1. Update the CI/CD Editor with:
include:
  - local: template2.yml

Observe the validation passes.

  1. When you commit and run the pipeline, you'll see this error:

Screenshot_2023-02-08_at_4.49.34_PM

Proposal

Investigate why the .gitlab-ci.yml file is included in the files list when validating on pipeline run but not when validating in the CI/CD Editor (or upon pipeline simulation). Update the validation behaviour so that the includes count is the same in both scenarios.

Findings

When the pipeline is run, we internally inject an include into the config content as explained here, causing the file count to increase by 1.

Implementation

First iteration enables us to determine whether an 'include' file was dumped into the config content internally. We will use this information in the fix for #367150 (closed) to adjust the file count accordingly.

backend MR
Expose when 'include' is internally injected into CI config content !112608 (merged)
Edited by Leaminn Ma