GitLab CI Recipes MVC
Problem to solve
We have a great feature for bootstrapping an entire .gitlab-ci.yml
file from CI templates. This is great if you want to create a brand new CI file. But what if I have a mostly working file, but just want to add one "feature" to it. For instance deploy to Heroku or run unit tests or call out to Twilio to text me about a failure.
Some customers are already building things like this with librarys of "DevOps" tasks in a /devops
repo with things like:
- This is how you run AWS CLI in GitLab CI/CD
- Here's the standard for calling Sonatype
- etc.
I think there is a space here for us to provide more convention and less configuration. For common places we integrate or common things we think people should run, we could provide smaller job
level templates (rather than the entire pipeline we provide today).
This is some ways is similar to what GitHub actions is providing, as they take the idea of "what do you want to do when building / testing / deploying" your code and break it into its component parts, and allow people to provide libraries of best practices and different ways of using a specific technology such as Node deploy to Azure, run some terraform, or even deploy a playlist to Spotify.
Further details
This can be done with AWS, Azure, GCP deployment steps, or things like Test Boosters.
Note - since snippets
is already a thing in GitLab, I actually don't think we should name this feature "CI Snippets". It was a convinent way of making my idea clearer. Other ideas for naming:
- CI fragments
- CI building blocks
Proposal
Provide job
level" templates accessable when building my CI yaml. Ideally these would be instance or group level configurable so as an admin I can publish my own along side the GitLab built in ones. This can be done in a clever way today using includes and extends:
Main .gitlab-ci.yml (in reality, the AWS_* variables would likely be secret project variables and not hardcoded here, but they are included for demonstrative purposes.)
include: recipes/aws_deployment.yaml
do_aws_deploy:
extends: .aws_deployment
variables:
- AWS_ACCESS_KEY_ID: xxx
- AWS_SECRET_ACCESS_KEY: xxx
The recipes/deploy_aws.yaml example would look something like this:
.aws_deployment:
image: amazon/aws-cli
script:
- long, resuable deployment script
stage: deploy
In this way we would end up with a syntax very similar to how the AWS GitHub action is invoked: https://github.com/actions/aws
action "Topic" {
uses = "actions/aws/cli@master"
args = "sns create-topic --name my-topic"
secrets = ["AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY"]
}
Discoverability of these could be somewhat poor, but that could be improved in a future iteration. You can imagine there being a "one click" button to add the include and extend the appropriate job, for example.
To start with, it would be great to provide three templates:
- AWS deploy
- GCP deploy
- Azure deploy
These would be analagous to the available GitHub actions for doing cloud deployments.
Atlassian Pipes
Buddy
They achieve this like this:
What does success look like, and how can we measure that?
Links / references
- https://techcrunch.com/2018/11/07/circleci-launches-orbs-a-package-manager-for-software-delivery-automation/
- https://github.com/features/actions (the marketplace)
This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.