<!--IssueSummary start--> <details> <summary> Everyone can contribute. [Help move this issue forward](https://handbook.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows/#contributor-links) while earning points, leveling up and collecting rewards. </summary> - [Close this issue](https://contributors.gitlab.com/manage-issue?action=close&projectId=278964&issueIid=345805) </details> <!--IssueSummary end--> ## Introduction First of all, I hope that, despite the long description, the answer will be "You are doing it wrong. Here is how you should do it". However, since I don't know better, I'll explain here with as many details I can provide. Testing changes in the CI when using included files, especially on more than one nesting level, is by far the most complicated and time-consuming process. It requires providing branch names almost everywhere, committing these changes, and removing them once I complete my changes. ### Scenario Below are a couple of YAML files taken from an actual project (just slightly sanitized). `my-group/my-front-end-project/.gitlab-ci.yml`: ```yaml --- include: - project: 'my-group/my-cd-project' file: - '/gitlab-ci/front-end.yml' ``` `my-group/my-cd-project/gitlab-ci/front-end.yml`: ```yaml --- image: some-image workflow: rules: - if: '$CI_COMMIT_MESSAGE =~ /-wip$/ || $CI_PIPELINE_SOURCE == "merge_request_event"' when: never - when: always stages: - development - prepare - quality - tests - qa - staging - production - documentation include: - project: 'my-group/my-cd-project' file: - '/gitlab-ci/global.yml' - '/gitlab-ci/before-script.yml' - '/gitlab-ci/after-script.yml' - '/gitlab-ci/prepare-cache.yml' - '/gitlab-ci/qa/duplication.yml' - '/gitlab-ci/integrations/build-succeeded.yml' pages: extends: .cache-pull stage: documentation script: - rm -rf public - npm run doc:generate -- --output public - echo "Pages URL $CI_PAGES_URL" artifacts: paths: - public rules: - if: '$CI_COMMIT_REF_SLUG == "master"' ``` I want to make some changes in the CI (on several of the included files) and test them without pushing them to the production branch. What I usually do is the following [^thruth-to-be-told]: 1. Create a new branch called `my-new-feature` in `my-group/my-front-end-project` 2. Create a new branch called `my-new-feature` in `my-group/my-cd-project` 3. In `my-group/my-front-end-project/.gitlab-ci.yml`, add a `ref: my-new-feature` to the `include` keyword 4. Do the same in `my-group/my-cd-project/gitlab-ci/front-end.yml` 5. Do the same in all the files included in `my-group/my-cd-project/gitlab-ci/front-end.yml` if they also include files. 6. Repeat as long as there are nested included files (and as long as I need to modify them). 7. Cry if the included files spread across other projects (this is the last step because, when it happens, I usually realize it after dealing with all the previous steps). [^thruth-to-be-told]: Frustration often leads me to skip all of this and push my changes to the main branch, which I hate to do. ### Proposal Define "root" variables that tell the runner to include files from a branch with a given name but fall back to another one, should that branch not exist. E.g.: ``` --- variables: INCLUDE_FROM_REF: $CI_COMMIT_REF_SLUG INCLUDE_FROM_REF_FALLBACK: $CI_DEFAULT_BRANCH ``` On every include, the runner will first try to include the file from `INCLUDE_FROM_REF` and: 1. Try to include the file from `INCLUDE_FROM_REF`. 2. Fail with an error if there is no branch with this name and no provided `INCLUDE_FROM_REF_FALLBACK`. 3. If the previous step didn't fail, try to include the file from `INCLUDE_FROM_REF_FALLBACK`. 4. Fail with an error if there is no `INCLUDE_FROM_REF_FALLBACK` branch. I imagine that the CI Linter could handle the failure, as the required branches are meant to exist before starting any pipeline.