gitlab-ci.yml include from submodule

Problem to solve

https://docs.gitlab.com/ee/ci/yaml/#include

Is very useful feature, as pipeline is more and more used between projects, developers want to factorise their pipeline logic.

As per documentation,

Note: We don’t support the inclusion of local files through Git submodules paths.

Further details

The other way of pipeline factorisation between project is external remote URL. Issues with this method is:

  • Security: no access control is possible for build pipeline, so we need to put the file freely available near the gitlab server.
  • Reproducibility: if the remote file is updated, rebuilding old versions is not ensured to work.

Having a build submodule pattern would allow to store more than the gitlab-ci.yml in the same workspace, with source control.

Proposal

This issue is to track support of submodule path include feature.

I see two options:

  • Either we implement this specifically in the gitlab-ci.yml include logic
  • Implement submodule traversal in gitaly, which would also benefit other features like raw blob access.

What does success look like, and how can we measure that?

local include form will check for submodules in the path and traverse them automatically.

Links / references