Recursive includes should allow duplicate and overwrite the last one or set a flag to allow duplicates

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

Summary

I'm having an issue with recursive includes where I cannot do diamond-like includes because of restriction on duplicates.

There should be a flag to indicate that I want to allow duplicates.

Steps to Reproduce:

I have a .base.yml file that is included by .sub-base1.yml and .sub-base2.yml. Both .sub-base1.yml and .sub-base2.yml are included in a pipeline but this does not work as I get hit by the duplicate file error (https://gitlab.com/gitlab-org/gitlab-ee/blob/master/lib/gitlab/ci/config/external/mapper.rb#L78).

       base.yml
       /      \
 base1.yml   base2.yml
       \      /
       pipeline (fail with duplicate include base.yml)

What is the expected correct behavior

I expect to be able to include base files that might include the same base.

Propose solution

I propose a flag or variable called ALLOW_DUPLICATE_INCLUDES (or whatever name you like): true/false (default false) that can be set to allow this scenario and let many people make DRYer ci files.