Use cloud native buildpacks for Auto DevOps
Problem to solve
Cloud Native Buildpacks (CNB) recently joined the CNCF, with involvement from Heroku and Pivotal:
- https://www.cncf.io/blog/2018/10/03/cncf-to-host-cloud-native-buildpacks-in-the-sandbox/
- https://github.com/buildpack
- https://buildpacks.io/
CNB aims to provide a standard mechanism for detecting code and producing a standards-based (https://www.opencontainers.org compliant) container runtime. The benefit of using buildpacks over a fully-implemented CI/CD pipeline encoded in yaml or some other format is that this creates a separation between development and operational execution of the container runtime. In short, developers can build applications in a general way, without being overly aware of operational concerns and best practices. As soon as code is checked in, a buildpack can run which detects the code in the repository and automatically builds an OCI image. This can then automatically be picked up and flow through the path to production according to pre-set rules and requirements.
We have some of this today. We are already using Heroku buildpacks in CI and AutoDevOps/CD are already using containers to orchestrate deployments in GKE. Using and contributing to this project instead of building our own solution benefits us in a few ways:
- Supporting CNCF/Open Source in general
- Container becomes natural handoff point between CI and CD to avoid blurring concerns
- Not having to invent our own separate solution
- Leverage buildpacks being developed and improved upon by many open source contributors
- Better interactivity with other products using compatible solutions
- Opportunity, if involved early, to help shape the project
The state of the project at the moment is that there is a specification and reference implementation, but neither Heroku or Pivotal have introduced the feature in their commercial products.
Target audience
- For Sasha (Developer), buildpacks in GitLab would provide assurance that software is being built and deployed with best practices built in; Sasha can focus on features and requirements in the app itself, and not worry about how it's going to be packaged up and delivered to different environments.
- Devon (DevOps Engineer) benefits by knowing that developers are using a proven, industry standard approach for generating containers to be deployed. Containers generated by development teams can be more trusted since they are more consistent.
Further details
This requires work on the CI side as well as the CD side.
- CI should be updated to prefer using CNB with the primary output of CI being an OCI image and associated metadata.
- The deployment portions of AutoDevOps should be rewired to take advantage of additional operational capabilities provided by OCI/buildpacks.
There is a v3 lifecycle specification available at https://github.com/buildpack/spec and a reference implementation at https://github.com/buildpack/lifecycle.
Proposal
Update CI/CD (AutoDevOps) to follow v3 CNB lifecycle which maps to GitLab as follows, and where everything in that flow beyond Git/Code is handled by GitLab automatically using smart defaults/predictable behavior. Infrastructure (owned by ~Configure) is implicit above somewhere between container and cluster.
Git/Code -(CI BuildPack)-> Container -(CD AutoDevOps)-> Cluster
We can also:
- Consider contributing to CNCF project
- Consider sponsoring CNCF project (not sure if this is possible?)
What does success look like, and how can we measure that?
Measure adoption of CNB in CI/CD pipelines
Links / references
- https://www.cncf.io/blog/2018/10/03/cncf-to-host-cloud-native-buildpacks-in-the-sandbox/
- https://github.com/buildpack
- https://buildpacks.io/
- https://github.com/buildpack/spec
- https://github.com/buildpack/lifecycle
This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.