Include predefined templates in .gitlab-ci.yml files
Problem to solve
Following discussions on https://gitlab.com/gitlab-org/gitlab-ee/issues/6167 and https://gitlab.com/gitlab-org/gitlab-ee/issues/5613, this issue tightens up the two by defining the final requirement: include external vendored files.
Further details
Security Products have a release and distribution by nature, because the tools are not shipping as part of GitLab (https://gitlab.com/gitlab-org/security-products/release/blob/master/docs/release_process.md). They are provided as docker images to be downloaded for each job run. This makes versioning particularly complex and tedious. The best and easiest way to deal with the job definitions is to provide them as vendored template, shipped with each version of GitLab. GitLab-CI already supports external files includes (https://docs.gitlab.com/ee/ci/yaml/#include). Nonetheless, the syntax does not allow includes of files present in the local instance.
Proposal
Create a new sub-folder in https://gitlab.com/gitlab-org/gitlab-ce/tree/master/lib/gitlab/ci/templates
for security jobs templates (like SAST, DAST, etc.).
Include templates for security features snippets to that folder: templates should be self-contained so they can be used as independent .gitlab-ci.yml
files.
This sub-folder content will be nicely shown for .gitlab-ci.yml
templates when adding a new .gitlab-ci.yml
via GitLab, thus making it also possible to discover.
Support a new syntax in .gitlab-ci.yml
to include predefined templates:
Track the usage (see below).
include:
- template: SAST
This will load SAST.gitlab-ci.yml
file, with the same rules defined in https://docs.gitlab.com/ee/ci/yaml/#include.
These are additional benefits:
- This will work for any template file included in GitLab (not just security)
- We can create job definition templates
- We can include them in a
.gitlab-ci.yml
file - Linting the file is not reporting any error
- The included file content is run as part of the pipeline
What does success look like, and how can we measure that?
We can measure the usage by tracking how many times the template:
keyword is parsed.