Spike: Idea to Parallelize Omnibus Building and Reduce Omnibus Upgrade Diffs
Spike final outcome
The work and discoveries from this spike produced the plan that will be executed in Distributed Asynchronous Builds in Omnibus GitLab (gitlab-org&12054)
Assumptions Describing the Problem Set
- Omnibus builds are becoming too slow for efficient testing
- Customers have to download the entire Omnibus package every time for upgrades which, though good for AirGap environments is not good for other environments and costs us in bandwidth
- Pi Builds are becoming problematic due to build times
Proposed Solution
- Break Omnibus builds up by dependency and parallelize the builds
- Ship Omnibus both as a single package (easy for airgaps) and as a meta-package that knows the dependencies and installs them all
Iterative Steps
The first step would be to take a single build and as a proof of concept show that we can do a build using our simple build method for testing software dependencies and deliver that as an artifact package to the next pipeline.
If this works for us, then we can expand this approach and parallelize all builds with our existing Omnibus software definitions. This could then be transitioned in a final step to create the meta-package.
Edited by Robert Marshall