Skip to content

GitLab Next

Closed (moved)
Created by Brendan O'Leary 🐢@brendanDeveloper

GitLab CI Recipes

Problem to solve

We have a great feature for bootstrapping an entire .gitlab-ci.yml file from CI templates. This is great if you want to create a brand new CI file. But what if I have a mostly working file, but just want to add one "feature" to it. For instance deploy to Heroku or run unit tests or call out to Twilio to text me about a failure.

Some customers are already building things like this with librarys of "DevOps" tasks in a /devops repo with things like:

  • This is how you run AWS CLI in GitLab CI/CD
  • Here's the standard for calling Sonatype
  • etc.

I think there is a space here for us to provide more convention and less configuration. For common places we integrate or common things we think people should run, we could provide smaller job level templates (rather than the entire pipeline we provide today).

This is some ways is similar to what GitHub actions is providing, as they take the idea of "what do you want to do when building / testing / deploying" your code and break it into its component parts, and allow people to provide libraries of best practices and different ways of using a specific technology such as Node deploy to Azure, run some terraform, or even deploy a playlist to Spotify.

Further details

This can be done with AWS, Azure, GCP deployment steps, or things like Test Boosters.

Note - since snippets is already a thing in GitLab, I actually don't think we should name this feature "CI Snippets". It was a convinent way of making my idea clearer. Other ideas for naming:

  • CI fragments
  • CI building blocks

Proposal

Provide job level" templates accessable when building my CI yaml. Ideally these would be instance or group level configurable so as an admin I can publish my own along side the GitLab built in ones. This can be done in a clever way today using includes and extends:

Main .gitlab-ci.yml (in reality, the AWS_* variables would likely be secret project variables and not hardcoded here, but they are included for demonstrative purposes.)

include: recipes/aws_deployment.yaml

do_aws_deploy:
  extends: .aws_deployment
  variables:
    - AWS_ACCESS_KEY_ID: xxx
    - AWS_SECRET_ACCESS_KEY: xxx

The recipes/deploy_aws.yaml example would look something like this:

.aws_deployment:
  image: amazon/aws-cli
  script: 
    - long, resuable deployment script
  stage: deploy

In this way we would end up with a syntax very similar to how the AWS GitHub action is invoked: https://github.com/actions/aws

action "Topic" {
  uses = "actions/aws/cli@master"
  args = "sns create-topic --name my-topic"
  secrets = ["AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY"]
}

Discoverability of these could be somewhat poor, but that could be improved in a future iteration. You can imagine there being a "one click" button to add the include and extend the appropriate job, for example.

To start with, it would be great to provide three templates:

  • AWS deploy
  • GCP deploy
  • Azure deploy

These would be analagous to the available GitHub actions for doing cloud deployments.

Atlassian Pipes

Buddy

https://buddy.works/

They achieve this like this:

image

What does success look like, and how can we measure that?

Links / references

Edited by Jason Yavorska