Include predefined templates in .gitlab-ci.yml files
Problem to solve
Following discussions on https://gitlab.com/gitlab-org/gitlab-ee/issues/6167 and https://gitlab.com/gitlab-org/gitlab-ee/issues/5613, this issue tightens up the two by defining the final requirement: include external vendored files.
Security Products have a release and distribution by nature, because the tools are not shipping as part of GitLab (https://gitlab.com/gitlab-org/security-products/release/blob/master/docs/release_process.md). They are provided as docker images to be downloaded for each job run. This makes versioning particularly complex and tedious. The best and easiest way to deal with the job definitions is to provide them as vendored template, shipped with each version of GitLab. GitLab-CI already supports external files includes (https://docs.gitlab.com/ee/ci/yaml/#include). Nonetheless, the syntax does not allow includes of files present in the local instance.
Create a new sub-folder in
https://gitlab.com/gitlab-org/gitlab-ce/tree/master/lib/gitlab/ci/templates for security jobs templates (like SAST, DAST, etc.).
Include templates for security features snippets to that folder: templates should be self-contained so they can be used as independent
This sub-folder content will be nicely shown for
.gitlab-ci.yml templates when adding a new
.gitlab-ci.yml via GitLab, thus making it also possible to discover.
Support a new syntax in
.gitlab-ci.yml to include predefined templates:
Track the usage (see below).
include: - template: SAST
This will load
SAST.gitlab-ci.yml file, with the same rules defined in https://docs.gitlab.com/ee/ci/yaml/#include.
These are additional benefits:
- This will work for any template file included in GitLab (not just security)
- We can create job definition templates
- We can include them in a
- Linting the file is not reporting any error
- The included file content is run as part of the pipeline
What does success look like, and how can we measure that?
We can measure the usage by tracking how many times the
template: keyword is parsed.