AutoDeploy gitlab components
Promoted to epic &99
With AutoDeploy deploy, we started a journey toward continuous delivery of gitlab.com, but there are several components bundled with gitlab that still needs a lot of manual work to be part of a release.
gitaly
, gitlab-pages
, gitlab-workhorse
, gitlab-shell
, and gitlab-elasticsearch-indexer
are based on version files committed in gitlab and omnibus.
As an engineer, to get a feature shipped, you have to:
- open a MR on the component project and get it merged
- ask a maintainer to cut a release
- wait for a release
- create a MR on gitlab-org/gitlab (i.e., gitlab-org/gitlab!17690 (diffs)) to upgrade the version file and get it merged
Proposed solution
In gitlab-org/release-tools!716 (merged) I'm proposing an alternative that allows the continuous deployment of gitlab-pages leveraging AutoDeploy
Instead of waiting for releases to be cut on the component project, we can apply the same pattern we have on auto-deploy, searching for the last green pipeline on the component project.
Then we update the version files on the auto-deploy branch.
Because those projects usually move slower than the rails codebase, we can pick master
version instead of working with auto-deploy branches in the project.
It can be easily extended to support other projects, in gitlab-org/release-tools!728 (merged) I show how to auto-deploy gitlab-workhorse
, gitlab-shell
, and gitaly
.
There's a drawback here, in auto-deploy we lose semver for components, they are selected based on commit SHA.
Impact
This change affects the whole engineering department bringing component development on par with the main rails codebase.
It is also beneficial for our plan to move security development to gitlab.com (gitlab-org/release&14 (closed)).
Today, without automation, the 4 steps above must be executed up to 4 times (master + 3 backports) blocking an engineer and a maintainer for a lot of time.
With this, we could aim to keep auto-deploy running on gitlab.com, including during security releases, without disclosing any information.
Open questions
We have to identify components that are tested end-to-end in CI.
Those already tested are good candidates for auto deploy.
component | tested in GitLab | tested in GitLab-QA | issue |
---|---|---|---|
GitLab-Pages | no | no | gitlab-org/gitlab-qa#411 |
GitLab-Workhorse | no | yes | |
GitLab-Shell | no | yes | |
Gitaly | yes | yes | |
GitLab-elasticsearch-indexer | yes | ??? |
/cc @gitlab-org/delivery @zj-gitlab @nick.thomas @jacobvosmaer-gitlab