Include external vendored files

Problem to solve

Following discussions on https://gitlab.com/gitlab-org/gitlab-ee/issues/6167 and https://gitlab.com/gitlab-org/gitlab-ee/issues/5613, this issue tightens up the two by defining the final requirement: include external vendored files.

Further details

Security Products have a release and distribution by nature, because the tools are not shipping as part of GitLab (https://gitlab.com/gitlab-org/security-products/release/blob/master/docs/release_process.md). They are provided as docker images to be downloaded for each job run. This makes versioning particularly complex and tedious. The best and easiest way to deal with the job definitions is to provide them as vendored template, shipped with each version of GitLab. GitLab-CI already supports external files includes (https://docs.gitlab.com/ee/ci/yaml/#include). Nonetheless, the syntax does not allow includes of files present in the local instance.

Proposal

Add a new syntax for vendored files:

# Content of .gitlab-ci.yml

include: ~'sast.yml'

rspec:
  script:
    - bundle exec rspec

As the current specification of includes takes a file path or a URL, we would need a different syntax to indicate we're using a vendored file of the instance. Alternatively include: @'sast.yml' could be used, or any other significative char.

The vendored files will be located in a template folder (to be determined), part of GitLab files tree.

What does success look like, and how can we measure that?

• We can vendor job definition templates
• We can include them in a .gitlab-ci.yml file
• Linting the file is not reporting any error
• The included file content is run as part of the pipeline

Links / references