Include external files in .gitlab-ci.yml
Description
When companies have many projects, they often have standardized testing and deploying processes, which are often reflected in .gitlab-ci.yml
. We should make it easy to share these processes and DRY up CI/CD configuration by letting one project import/include from another project's repo; either its .gitlab-ci.yml
or another specified file. By allowing multiple imports, configuration could be componentized with a mix-and-match approach to constructing .gitlab-ci.yml
content, possibly using YML templates defined in the included file.
Since this is targeting larger organizations, it will be ~"EE Premium".
Proposal
Add a new include
keyword in .gitlab-ci.yml
that allows inclusion of external definitions.
The file can be:
- local to the same repository, referenced using the path
- remote in a different location, accessed using HTTP(S) protocol, referenced using the full URL
- This can be used at the top level of the file, not inside any section or job definition
- Includes are done first, the resulting file is composed, then it is evaluated as it were a single local file
- Support for multiple
include
can be done using an array instead of a single value
Still to be defined / possible improvements
- Nested use of
include
(keyword used in included files too) - Support for HTTP(S) authentication
- Relative/absolute local paths
- Lint support for included files
- Behavior if used in ~CE or ~"EE Starter"
Examples
Import a definition from another file in the same project:
include: /templates/.gitlab-ci-templates.yml
Import a definition from another project:
include: https://gitlab.com/same-group/another-project/raw/master/.gitlab-ci-templates.yml
Include a specific version of the file:
include: https://gitlab.com/gitlab-org/gitlab-ce/blob/8efdf75b/.gitlab-ci-templates.yml
Use template definitions:
include: https://gitlab.com/same-group/another-project/raw/master/.gitlab-ci-templates.yml
test:
<<: *predefined_test
Out of scope
Mix-and-match raw (direct import into job):
test1:
include: same-group/template-repo/.gitlab-ci-rspec.yml
test2:
include: same-group/template-repo/.gitlab-ci-rubocop.yml
This last option might seem easier because you can ignore YAML templates, which are pretty awkward. But you quickly end up including the same file multiple times, buried deep in the config file, and that feels like an anti-pattern. Dropping this scope may also ensure that each included YAML file can be fully validated on its own.
Links
- Requested here: https://forum.gitlab.com/t/ci-common-configuration/3862
- Older proposal: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/1258
Original proposal
Use a fully-defined pipeline from another project:
include: same-group/another-project/.gitlab-ci.yml
Use template definitions:
include: same-group/template-repo/.gitlab-ci.yml
test1:
<<: *predefined_template
Mix-and-match template definitions:
include:
- same-group/template-repo/.gitlab-ci-rspec.yml
- same-group/template-repo/.gitlab-ci-rubocop.yml
test1:
<<: *rspec_template
test2:
<<: *rubocop_template
Include from arbitrary file, URL, or specific branch:
include:
- group/project/directory/file.ext
- user/project/directory/file.ext
- https://server/url
- group/project#branch
Using predefined templates seems like the recommended best practice, but the first way is a great way to get started, or use if a company has a rigid reusable process. It's comparable to including all of Twitter bootstrap vs including only the components you use.