Ability to include versioned templates by specifying semantic ranges
Problem to solve
As a Platform Engineers I want to be able to granularly control minor & patch (see semver.org) template updates I push to my platform users so I can prevent breaking them when breaking changes occur.
Intended users
The major intended user will be Priyanka the Platform Engineer, with the beneficiaries being any consumers of shared CI templates.
User experience goal
An advantage of Gitlab CI template is the ability for customers to be able to customize them. This is a blight for the Platform Engineer as they worry about pushing updates to shared templates for fear of breaking dependent repos without an easy mechanisms for teams to pin historic versions (as standard package managers allow) themselves. They normally have to resort to finding the prior git commit
hash and including that version instead. It works but is inelegant beyond a certain scale.
By allowing platform engineers to just focus on the latest revision on major branches (ie. 1.x, 2.x) of shared templates it would simplify their development flow as they no longer have to worry about breaking clients as they have a way to insulate themselves if needed.
Proposal
Quite a nascent idea but potentially by extending the existing include
syntax to add a repo
option:
include:
- repo: artifactory.repo/gitlab-templates/my-shared-template-1
version: >=2.0.0 # Awesome if it'd support things like tilde/caret formats: https://nodesource.com/blog/semver-tilde-and-caret/
Really the repo
behavior should be viewed a customization atop the existing include.remote
behavior where it can compute parts of the URL before it fetches it. I'd also be open to some templating system if this is a potentially more constructive path forward.
I've never actually implemented a repository format before but guessing that artifacts themselves we'd expect to be published in some pattern or the repo provider exposes some index that Gitlab would be able to use to resolve the version range specifier.
Out of scope
Likely out-of-scope would be nested, shared templates. I'm assuming that all the templates published to the artifact repository would be included within that artifact, either as a single file or even better if we could interpolate a tarball to act a little local context even better!
Further details
Coming from a Machine Learning platform team where no-one really knows how to satisfy all the CICD solutions Applied Scientists can think of to arrange their models, it has meant needing to take an iterative approach to development. Unfortunately this has created maintenance friction trying to satisfy wildly disparate pipeline needs that may only become apparent after a few months of iteration (meaning you made both problems use the same template to begin with).
The reason I'm recommending this as a top-level new feature is that we could do this using parent-child pipelines (ie. the parent does the version resolution and the child is the resolved pipeline). With that approach I think we'd lose the ability for projects themselves to customize the end-result as they'd need to override items in the child pipeline which I don't believe is currently possible.
Permissions and Security
Don't think this would require any additional permissions or security I guess other than being able to access the repository that holds the shared templates potentially but maybe that can be outside of scope? Our Artifactory has broad read-only access so likely wouldn't be needed for us at least.
Documentation
Availability & Testing
What does success look like, and how can we measure that?
What is the type of buyer?
Likely an enterprise tier feature, see limited appeal.
Is this a cross-stage feature?
Links / references
Overlaps with: