<!--IssueSummary start--> <details> <summary> Everyone can contribute. [Help move this issue forward](https://handbook.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows/#contributor-links) while earning points, leveling up and collecting rewards. </summary> - [Close this issue](https://contributors.gitlab.com/manage-issue?action=close&projectId=278964&issueIid=27896) </details> <!--IssueSummary end--> ### Problem to solve With the implementation of [Composable Auto DevOps](https://gitlab.com/gitlab-org/gitlab-ce/issues/47234) some job definitions have been placed in a `Jobs/` subfolder: https://gitlab.com/gitlab-org/gitlab-ee/tree/master/lib/gitlab/ci/templates/Jobs This has caused few troubles which demonstrate the need to enforce some rules (with tests) and provide guidelines for other teams to leverage the include feature. ### Intended users Developers ### Further details Here are assumptions from the experience of the ~Secure team when [Vendoring the template for Security Products](https://gitlab.com/groups/gitlab-org/-/epics/581) that need to be confirmed or adjusted: 1. Template name must be unique, whatever the subfolder it is placed in. E.g. having these two templates should result in a test failure: - `Security/DAST.gitlab-ci.yml` - `Jobs/DAST.gitlab-ci.yml` This rule was motivated by the behavior explained in `#2`. 2. Template placed in a subfolder can be included without specifying the folder. E.G ```yaml include: - template: SAST.gitlab-ci.yml ``` The gotcha we just discovered is that for `#2` to work, [the folder actually has to be listed as a category](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/template/gitlab_ci_yml_template.rb#L16-22). For `Security` templates, this was done [in EE only by extending the list](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/lib/ee/gitlab/template/gitlab_ci_yml_template.rb#L19-23). Though having the folder listed as a category also has the side effect to display the templates it contains in the Template dropdown in GitLab UI. For this reason (AFAIR), the `Jobs` template where not listed as a category. The resulting behavior is: - in CE, `Security` templates must be included using the folder prefix: `template: Security/SAST.gitlab-ci.yml` (though, we don't particularly want to include them in CE, all our features are premium tiers) - in EE, `Security` templates can be included both with or without the folder prefix: `template: Security/SAST.gitlab-ci.yml` or `template: SAST.gitlab-ci.yml` - in CE and EE, `Jobs` templates must be included using the folder prefix: `template: Jobs/Build.gitlab-ci.yml` NB: The ~Secure feature have been vendored in `11.9` following these assumptions listed above and we need to keep them working as is to avoid breaking customer configs. At least, if we want to change the behavior, we need to deprecate it now and drop support with `12.0`. ### Proposal **As an urgent fix (before April 7th)** The `Jobs/DAST.gitlab-ci.yml` must be removed to only keep the existing `Security/SAST.gitlab-ci.yml`. This is addressed by this MR: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/26957 **As a short term solution (before April 22nd?/release of `11.10`)** We need to [move Security templates into Jobs subfolder](https://gitlab.com/gitlab-org/gitlab-ce/issues/59986) to benefit from [further improvements regarding job templates](https://gitlab.com/gitlab-org/gitlab-ce/issues/53307) We need to decide which behavior we want to provide and align all our job templates on it. Do we want to keep allowing inclusion of templates without specifying the subfolder? - YES: - rule `#1` must be enforced, by adding a unit test ensuring uniqueness of template names, whatever their subfolder (both in CE and EE), and document this. - NO: - we need to announce deprecation of what ~Secure has released in `11.9`, which is using unprefixed include e.g. `template: SAST.gitlab-ci.yml` and promote prefixed usage everywhere. Though, to avoid redoing a change later, I'd highly recommend that instead of asking to use the `Security/` prefix we actually [move Security templates into Jobs subfolder](https://gitlab.com/gitlab-org/gitlab-ce/issues/59986) directly. - we need to decouple the logic that allows unprefixed include from the one that exposes the templates in the UI (dropdown), so that without exposing the `Jobs/` templates in the UI, we still allow to include them unprefixed (to avoid breaking change for what ~Secure has released in `11.9`). This code could then be dropped in `12.0` to disallow unprefixed include, if not used anywhere else. ### What does success look like, and how can we measure that? Unified behavior for job templates, at least from `12.0` if it implies breaking changes for what has been already released. ### Links / references