Skip to content
GitLab
Next
Closed
Issue created by Robert Speicher@rspeicher

API-based Omnibus release during a security release pulled GitLab version from the wrong remote

The 13.3.3 security release was performed with the omnibus_api_release feature flag enabled, and we subsequently discovered that the 13.3.3 Omnibus packages used the 13.3.2 gitlab-rails version.

See the job output here: https://ops.gitlab.net/gitlab-org/release/tools/-/jobs/1745061

The GitlabEeRelease tags and pushes 13.3.3 to dev:

2020-09-02 09:27:21.247658 I ReleaseTools -- Starting EE release -- {:version=>"13.3.3-ee"}
2020-09-02 09:27:21.247741 I ReleaseTools::Release::GitlabEeRelease -- Preparing repository...
2020-09-02 09:29:17.600704 I ReleaseTools::Release::GitlabEeRelease -- Compiling changelog -- {:version=>"13.3.3-ee"}
2020-09-02 09:31:57.480477 I ReleaseTools::Release::GitlabEeRelease -- Bumping version -- {:file_name=>"VERSION", :version=>"13.3.3-ee"}
2020-09-02 09:31:58.286163 I ReleaseTools::Release::GitlabEeRelease -- Pushing ref to remotes -- {:name=>"13-3-stable-ee", :remotes=>[:dev]}
2020-09-02 09:32:02.201099 I ReleaseTools::Release::GitlabEeRelease -- Pushing ref to remotes -- {:name=>"master", :remotes=>[:dev]}
2020-09-02 09:32:05.965587 I ReleaseTools::Release::GitlabEeRelease -- Creating tag -- {:name=>"v13.3.3-ee"}
2020-09-02 09:32:05.973551 I ReleaseTools::Release::GitlabEeRelease -- Pushing ref to remotes -- {:name=>"v13.3.3-ee", :remotes=>[:dev]}

Then, the Omnibus release fetches the gitlab-rails VERSION file from 13-3-stable-ee on Security, which returns 13.3.2:

2020-09-02 09:32:22.356152 I ReleaseTools::PublicRelease::OmnibusGitlabRelease -- Creating EE tag -- {:tag=>"13.3.3+ee.0", :project=>"gitlab-org/security/gitlab"}
2020-09-02 09:32:23.000934 D ReleaseTools::GitlabClient -- [HTTParty] [2020-09-02 09:32:23 +0000] 404 "GET https://gitlab.com/api/v4/projects/gitlab%2Dorg%2Fsecurity%2Fomnibus%2Dgitlab/repository/tags/13%2E3%2E3%2Bee%2E0" 31 
2020-09-02 09:32:23.181432 D ReleaseTools::GitlabClient -- [HTTParty] [2020-09-02 09:32:23 +0000] 200 "GET https://gitlab.com/api/v4/projects/gitlab%2Dorg%2Fsecurity%2Fgitlab/repository/files/VERSION/raw" 10 
2020-09-02 09:32:23.354510 D ReleaseTools::GitlabClient -- [HTTParty] [2020-09-02 09:32:23 +0000] 200 "GET https://gitlab.com/api/v4/projects/gitlab%2Dorg%2Fsecurity%2Fomnibus%2Dgitlab/repository/files/VERSION/raw" 6

Immediately there are two possible corrective actions I can think of, and we may want to do both just for extra precaution:

  1. Update the existing on-disk releases for GitlabEe/GitlabCe to push to both Security and Dev. => gitlab-org/release-tools!1167 (merged)
  2. Update all of the API-based releases to always use dev_path.

cc @gitlab-org/delivery

Edited by Robert Speicher