API-based Omnibus release during a security release pulled GitLab version from the wrong remote
The 13.3.3
security release was performed with the omnibus_api_release
feature flag enabled, and we subsequently discovered that the 13.3.3 Omnibus packages used the 13.3.2 gitlab-rails version.
See the job output here: https://ops.gitlab.net/gitlab-org/release/tools/-/jobs/1745061
The GitlabEeRelease
tags and pushes 13.3.3 to dev
:
2020-09-02 09:27:21.247658 I ReleaseTools -- Starting EE release -- {:version=>"13.3.3-ee"}
2020-09-02 09:27:21.247741 I ReleaseTools::Release::GitlabEeRelease -- Preparing repository...
2020-09-02 09:29:17.600704 I ReleaseTools::Release::GitlabEeRelease -- Compiling changelog -- {:version=>"13.3.3-ee"}
2020-09-02 09:31:57.480477 I ReleaseTools::Release::GitlabEeRelease -- Bumping version -- {:file_name=>"VERSION", :version=>"13.3.3-ee"}
2020-09-02 09:31:58.286163 I ReleaseTools::Release::GitlabEeRelease -- Pushing ref to remotes -- {:name=>"13-3-stable-ee", :remotes=>[:dev]}
2020-09-02 09:32:02.201099 I ReleaseTools::Release::GitlabEeRelease -- Pushing ref to remotes -- {:name=>"master", :remotes=>[:dev]}
2020-09-02 09:32:05.965587 I ReleaseTools::Release::GitlabEeRelease -- Creating tag -- {:name=>"v13.3.3-ee"}
2020-09-02 09:32:05.973551 I ReleaseTools::Release::GitlabEeRelease -- Pushing ref to remotes -- {:name=>"v13.3.3-ee", :remotes=>[:dev]}
Then, the Omnibus release fetches the gitlab-rails VERSION
file from 13-3-stable-ee
on Security, which returns 13.3.2
:
2020-09-02 09:32:22.356152 I ReleaseTools::PublicRelease::OmnibusGitlabRelease -- Creating EE tag -- {:tag=>"13.3.3+ee.0", :project=>"gitlab-org/security/gitlab"}
2020-09-02 09:32:23.000934 D ReleaseTools::GitlabClient -- [HTTParty] [2020-09-02 09:32:23 +0000] 404 "GET https://gitlab.com/api/v4/projects/gitlab%2Dorg%2Fsecurity%2Fomnibus%2Dgitlab/repository/tags/13%2E3%2E3%2Bee%2E0" 31
2020-09-02 09:32:23.181432 D ReleaseTools::GitlabClient -- [HTTParty] [2020-09-02 09:32:23 +0000] 200 "GET https://gitlab.com/api/v4/projects/gitlab%2Dorg%2Fsecurity%2Fgitlab/repository/files/VERSION/raw" 10
2020-09-02 09:32:23.354510 D ReleaseTools::GitlabClient -- [HTTParty] [2020-09-02 09:32:23 +0000] 200 "GET https://gitlab.com/api/v4/projects/gitlab%2Dorg%2Fsecurity%2Fomnibus%2Dgitlab/repository/files/VERSION/raw" 6
Immediately there are two possible corrective actions I can think of, and we may want to do both just for extra precaution:
- Update the existing on-disk releases for GitlabEe/GitlabCe to push to both Security and Dev. => gitlab-org/release-tools!1167 (merged)
- Update all of the API-based releases to always use
dev_path
.
Edited by Robert Speicher