Establish guidelines for CI/CD template submissions from Technology Partners

Proposal

Based on this MR: !51415 (closed) and later discussions in Slack, it seems we'll have a new "class" of CI/CD templates: CI/CD templates submitted by GitLab Technology Partners: https://about.gitlab.com/partners/technology-partners/

I believe we need to establish guidelines for every step of this process. Including:

1. Guidelines for creating the template itself, with standards that can be used across all templates. This includes:
   1. Where to put the template, and naming standards?
   2. What syntax is recommended, and what syntax to avoid?
   3. Required style?
   4. Required comments? For example, explain if it's intended to be added with include or used standalone. Explanation of variables, prerequisites, etc.
2. MR review process:
   1. Who to ping first?
   2. How to demonstrate the template in action?
3. Template maintenance:
   1. Who is responsible for maintenance?
   2. How is the template maintained? If there are issues, who should the public contact, and how should they do it?
   3. What happens to the template if the partnership expires?

Current Documentation

The current template guide is generally for internal use: https://docs.gitlab.com/ee/development/cicd/templates.html

@DarwinJS wrote some details at https://gitlab.com/guided-explorations/ci-cd-plugin-extensions/design-guide-and-templates-for-ci-cd-extensions, but I don't see this as clear/precise enough for the authors/reviewers.

added GitLab Core Partner contribution Technical Writing citemplates devopsverify grouppipeline authoring labels

changed the description

added to epic &4858 (closed)

Just a note that the referenced project I stood up is in reality pre-pre-MVC-1. Just needed to get something down due to the every increasing requests - especially from partners who are experienced at building these CI integrations across all major players in the CI tooling space.

changed the description

changed the description

I'd like to get some more eyes on this. I mentioned this in slack, but probably should have pinged directly here. I am willing to write up an initial first draft of guidelines for external template contributions, but can't answer all the questions above.

I guess the key initial questions I'd like answered:

• Is it OK to add all these to lib/gitlab/ci/templates? (I think it's yes, but worth asking if we need a subfolder)
• How can we ensure a smooth and standardized review process?
  • Can we request that the partner demonstrates the template works in their own GitLab group/project, with a green pipeline, before pinging for a template review?
• After a technology partner's template is merged into the repo, what will the maintenance process be like?
  • How do we verify that someone submitting a fix is from the technology partner? Or is it that anyone touch these templates after they are merged?
  • Is it OK to merge template changes that are considered "breaking"?
  • If there are problems in the template, are contributors expected to contact GitLab with issues, or contact the technology partner?

cc list:

• From grouppipeline authoring @dhershkovitch @nadia_sotnikova -> There is a lot of work ongoing regarding templates in &4858 (closed)
• From ~"group::configure" @tkuah @nagyv-gitlab -> Any worries from an autodevops perspective?
• From citemplates @shinya.maeda @matteeyah -> Any thoughts regarding reviews/maintenance/standards?

@dhershkovitch - you are definitely the DRI here, excited to see this progress and how we can make a great experience of this!

@marcel.amirault - I know @brianwald has some thoughts on integrating partner CI extensions - so inviting in.

As far as I know the current processing logic for locating "templates" has this unusual behavior of ignoring folder structure, but I can't locate it now. So this means required, but manual name uniqueness among all the files in the entire tree. A hard ask for anyone - but especially a partner.

I also opened this issue tracking the benefits of carefully selected new name.

There are also technically beneficial reasons that a partner having an entire folder to themselves would be helpful (or sub-sub folders for huge cloud partners where we deal with individual teams). Among them would be to allow deep include structures so partners can include script library code using includes, rather than forcing a container implementation. A working example and the backing rationale for this idea is covered in Script Code Libraries In Pure GitLab CI YAML and Complexity Oriented Architecture Decision Tree: Building GitLab Reusable CI/CD Extensions

MVC: Templates Subdirectory for Partner Managed CI / CD Integrations

On the review process I think it should be a "Continuous Testing" project with scheduled build & test - not just a review - for Level 2 partners. So that the partner gets "shifted left" early warning when a change in GitLab or their side unexpectedly breaks the integration. I think this would be too human intensive to ask of the first level of basic partnership.

On the maintenance process - as alluded to in the above - I think this should be shifted left for parnership levels 2 and 3 - neither GitLab nor the partner should be waiting for trailing indicators of quality problems because that means many partners are already experiencing a problem.

One new thing this discussion might need to consider is partners who submit CI or CD extensions that they would like co-customers to depend directly on. This is what we do with our own CI/CD code managed as product known as AutoDevOps and our CI / CD competitors provide such a framework as well. So far we seem to conceive of this as "Grab and Customize" starting points. It does not have to be spec'ed for being built now - but it could inform decisions now - such as whether a new folder would help if this "depend upon" mode was on the far out road map.

@marcel.amirault Can you changed the unordered list into an ordered list so it's easier to reference?

• Is it OK to add all these to lib/gitlab/ci/templates? (I think it's yes, but worth asking if we need a subfolder)

Yes, both to keeping it there and having a subfolder.

• Can we request that the partner demonstrates the template works in their own GitLab group/project, with a green pipeline, before pinging for a template review?

Yes. Our existing citemplates maintainers can review it along with the Technical Writing team.

After a technology partner's template is merged into the repo, what will the maintenance process be like?

I would just make sure everyone is aware that this is not a GitLab maintained CI template and have the partner maintain it.

@marcel.amirault Thank you for creating an issue. I currently don't have enough time to fully walkthrough all of the context around this issue, but I'll leave a few important aspects.

License

All contents packaged in GitLab are licensed. If we apply the "MIT Expat" license to the content, we and any users also have a right to change the content. If the third party organization strictly wants to obtain the ownership of the content, they would need to explicitly define the license for the specific content.

Granting a third party for fully control (a part of) our codebase means we could also be vulnerable in some scenario. For example, if the content was introduced by a third party had a security vulnerability and caused a negative impact on active users in gitlab.com or on-premises instances, we don't have a right to fix the content nor reject it from our codebase.

I'd advise to loop in legal person from our organization as well. I could be wrong.

Marketplace

What's missing in our CI/CD templates is to allow any users/organizations to host their templates for end-users.

GitHub already has had market place for GitHub Actions. Each provided action defines the task spec, arguments, docker images, etc to provide the solutions for end-users. Third parties have a full control on their contents and actively maintain in order to drive the adoption of their technologies. To give you an rough idea, an individual action is more or less similar to our Job-level template. Ideally, we should provide multi-tenant CI/CD template registry and move all the lib/gitlab/ci/templates contents into gitlab.org namespace.

Few thoughts on this topic:

• Do we need to package the templates into GitLab? I feel like the concept of third-party templates/marketplace inherently suggests it's maintained outside of the application itself.
• What if we just had an external project on gitlab.com that hosted the templates and someone interesting including a job template would pull from an external remote? I recognize this would cause issues with airgapped/self-hosted but it's a vendor job that integrates with an external service anyway, it doesn't seem like a big issue - those who want to use them locally could also copy it down to their instance.

If we went this route.. then the question is: As a end-user, how can ensure I am pulling from the "Officially maintained third-party templates".

Perhaps we could add a key to includes: that enforces the path to the external repository where it's hosted.

For example,

include:
  - vendor: CompanyName
    template: some-vendor-job-template.yml

where the vendor key will always reference the externally managed project (i.e. https://gitlab.com/third-party-managed-jobs/{CompanyName}/...)

@brianwald Yes, we've had discussions about some type of marketplace for templates recently, and this is the perfect use case for that concept. I can't find an issue specifically about it, but it'll be part of &4858 (closed)

@marcel.amirault @shinya.maeda @matteeyah

• Do we need to package the templates into GitLab? I feel like the concept of third-party templates/marketplace inherently suggests it's maintained outside of the application itself.

@brianwald is on target. As is currently designed, GitLab CI templates functions first as a starting off point.

The introduction of includes.template has muddied the waters here where users can now rely on CI templates that previously had no guarantees of availability, compatibility, nor maintenance. We have code review guidelines to address backward compatibility as a step but I feel that is not enough.

As a measure until we have versioned CI templates, I will propose that we only accept new CI templates where :

• It is requested frequently, e.g Docker.
• It is meant as a means for further integration, e.g. Auto-DevOps.

Additionally, I propose that we establish which CI templates are:

• Actively maintained,
• Functions purely as a template, and should be used with include.template at your own risk.

@tkuah As a starting point, every template that should not be used with includes should have a note right at the top that says as much, perhaps? And the same for those specifically designed to be used with includes.

I like Marcel's proposal. We can start with documentation and start building some functionality into the product for this if needed.

As a starting point, every template that should not be used with includes should have a note right at the top that says as much, perhaps? And the same for those specifically designed to be used with includes.

@marcel.amirault Maybe something softer, like using includes at your own risk as these templates are subject to change even at minor versions ?

How about "Not designed or maintained to be an include"

Which is definitely true of any of those that aren't completely variable driven because they can't be used as includes anyway - you have to customize them to use them.

By adding "or maintained" you are implying that there is no "non breaking" guarantee when they peg to the current version.

(Later we need to discuss that it seems like we are implicitly thinking we have to guarantee not to break production with the latest commit of these - which is actually a DevOps versioning antipattern because the "latest" release of any dependency should only ever be consumed by the "Dev" environment of anyone depending on it - just like the entire rest of the software ecosystem.)

marked this issue as related to #320698 (closed)

changed the description

This is sort of tangential, sort of not. Quite a while back I suggested a search engine microformat be embedded in snippets so they could be located anywhere on any web server using google or a curator process we used to pull it into a list.

Maybe we could specify a metadata format in Markdown Frontmatter that would also serve as the "listing data". In the near term, google would pick up the burden of indexing for us.

If we eventually created a curation collection mechanism it would know which ones were found in our official locations and which ones in the wild - like this person's personal MarketPlace for GitLab CI components: https://r2devops.io/jobs/

Creating an MVC of the frontmatter in markdown would be very easy to do.

Here is the original concept described in the context of snippets - but it would be directly reusable for CI templates: MVC of Community CI Snippet Catalog via Google Microformats

The current template guide is generally for internal use: https://docs.gitlab.com/ee/development/cicd/templates.html

@marcel.amirault This development guide for templates, as with every development guide is for the wider community. So I will consider https://docs.gitlab.com/ee/development/cicd/templates.html valid for technology partners as well.

@dhershkovitch - if marketplace is right around the corner, I think perhaps which of the two modes are in the marketplace vision should be settled. I think many partners will be willing to use the model of directly depending on CI components because it is familiar from competitors and AutoDevOps and non-CI coding frameworks (like NPM packages).

Technology Partnerships will definitely be more strategic to partners if depending on CI extensions is available.

It definitely affects the technology partnership programs @mlebeau has been tasked with building out.

@DarwinJS agreed - this is a key aspect of the new tiered partnership leveling I'm proposing which would include integration complexity as a factor to distinguish "Registered" partners (those who are simply building a grab n' go template) from those partners who choose to build a more robust & dependable CI extension that would be partner-managed.

@dhershkovitch would be great to connect and discuss your vision of the marketplace so that we can align on other potential areas for tiering our partnership levels.

Meeting is on for Thursday, everyone is welcome to join

added sectionops label

mentioned in issue gitlab-org/quality/triage-reports#2341 (closed)

mentioned in merge request !56843 (merged)

mentioned in issue gitlab-org/quality/triage-reports#2409 (closed)

mentioned in issue #287805 (closed)

mentioned in issue gitlab-org/quality/triage-reports#2506 (closed)

@marcel.amirault we should consider adding to the guidelines some of the ideas @shinya.maeda mention in his comment

@dhershkovitch Thanks for the reminder, I've created #326789 (closed) to keep track.

mentioned in merge request !57785 (closed)

mentioned in issue #326298

mentioned in issue gitlab-org/quality/triage-reports#2576 (closed)

mentioned in issue gitlab-org/quality/triage-reports#2703 (closed)

mentioned in issue gitlab-org/quality/triage-reports#2780 (closed)

changed milestone to %Backlog

added 1 deleted label

added workflowproblem validation label

removed workflowproblem validation label

mentioned in issue #330181

marked this issue as related to #330181

mentioned in epic &6256 (closed)

cc: @nbadiey for visibility

mentioned in merge request !66593 (closed)

I'm going to close this as the guidelines have been expanded a lot with !56843 (merged) (and later !78403 (merged)), so I think the main purpose of this issue was resolved with that first MR. We can continue to iterate as needed.

mentioned in merge request !78403 (merged)

changed milestone to %13.11

added docsimprovement label

closed

mentioned in merge request !93574 (merged)

added Category:Pipeline Composition label