Skip to content

Update CI includes counting structure to include duplicates

Leaminn Ma requested to merge 367150-update-ci-max-includes into master

What does this MR do and why?

As a first iteration to resolve Backend: Prevent unlimited number of includes f... (#367150 - closed) and https://gitlab.com/gitlab-org/gitlab/-/issues/390545, this MR updates the structure we use to count include files so that duplicate files are accounted for.

Specifically, this MR changes expandset from a Set to an Array. As a compensation for counting duplicates, we are also increasing MAX_INCLUDES to 150.

The changes are made behind FF: ci_includes_count_duplicates

How to set up and validate locally

  1. Update both max includes values to 2 in lib/gitlab/ci/config/external/context.rb:12.
          MAX_INCLUDES = 2
          NEW_MAX_INCLUDES = 2
  1. Set up the following files in your project root.

File1: template1.yml

job1:
  script: exit 0

File2: template2.yml

include:
  - local: template1.yml
  - local: template1.yml
  - local: template1.yml
job2:
  script: exit 0
  1. Go to the CI/CD Editor and update the contents with:
include:
  - local: template2.yml

First observe that there is no validation error.

  1. Enable the ci_includes_count_duplicates feature flag.
Feature.enable('ci_includes_count_duplicates')

And observe that the config from Step 3 produces an error (after refreshing the Editor):

Screenshot_2023-02-08_at_4.42.04_PM

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #367150 (closed)

Edited by Leaminn Ma

Merge request reports