Skip to content

Refactor Omnibus check packages

Dat Tang requested to merge dattang/refactor-omnibus-packages-check into master

What does this MR do and why?

Describe in detail what your merge request does and why.

Content

  • Refactor Omnibus check packages

The class CheckPackagesPublished can be used to check any GitLab versions - monthly or patch. Furthermore, the similar approach can be used to check the packages after tagging - just needs to change from using PackagesStatus::Availability to PackagesStatus::Functionality. Because of that, this commit refactor the class CheckPackagesPublished to be able to check both tagging and publishing.

  • Convert CheckPackagesPublished to Services::OmnibusPackages::Base, with the error classes and check packages service defined by the subclass.
  • Create Services::OmnibusPackages::Tagging and Services::OmnibusPackages::Publishing as the subclasses of Services::OmnibusPackages::Base.
  • Write rspec accordingly.

This MR prepares the code for both tagging and publishing checks. For patch release, the only difference is that we need to check 3 versions, thus we add a generated pipeline. The rest can be reused from this MR.

Ref: gitlab-com/gl-infra/delivery#20193 (closed), gitlab-com/gl-infra/delivery#20425 (closed), gitlab-com/gl-infra/delivery#20430 (closed)

Testing

This is a refactoring MR, so I don't perform a full test as it was done in !3333 (merged) already. Here I do two simple tests to make sure the rake task is still working as expected:

When no package is found

bundle exec rake "monthly:verify:check_omnibus_packages_publishing"
2024-09-03 15:20:12.581878 T ReleaseTools::Feature -- enabled? -- {:feature=>:switch_to_main_branch}
2024-09-03 15:20:12.879924 I ReleaseTools::Services::OmnibusPackages::Publishing -- Verifying Omnibus CE/EE package publishing for 17.4.0
2024-09-03 15:20:12.879992 I ReleaseTools::Services::PackagesStatus::Availability -- Fetching pipelines for tag -- {:tag=>"17.4.0+ce.0"}
2024-09-03 15:20:13.430575 D ReleaseTools::GitlabDevClient -- [HTTParty] [2024-09-03 15:20:13 +0200] 200 "GET https://dev.gitlab.org/api/v4/projects/gitlab%2Fomnibus-gitlab/pipelines" 2 
2024-09-03 15:20:13.430657 I ReleaseTools::Services::PackagesStatus::Availability -- Fetching pipelines for tag -- {:tag=>"17.4.0+ee.0"}
2024-09-03 15:20:13.863444 D ReleaseTools::GitlabDevClient -- [HTTParty] [2024-09-03 15:20:13 +0200] 200 "GET https://dev.gitlab.org/api/v4/projects/gitlab%2Fomnibus-gitlab/pipelines" 2 
2024-09-03 15:20:13.864096 I ReleaseTools::Slack::ReleaseJobEndNotifier -- Posting slack message -- {:job_type=>"Verify package publishing", :status=>:failed}
2024-09-03 15:20:13.864539 T ReleaseTools::Slack::Message -- post -- {:channel=>"C0139MAV672", :text=>"Verify package publishing job failed, review the job log for details - http://test.com/", :unfurl_links=>false, :unfurl_media=>false, :blocks=>[{:type=>"section", :text=>{:type=>"mrkdwn", :text=>":release: :ci_failing: *Verify package publishing job failed, review the <http://test.com/|job log> for details*"}}, {:type=>"context", :elements=>[{"type"=>"mrkdwn", "text"=>":clock1: 2024-09-03 13:20 UTC"}]}]}
2024-09-03 15:20:13.864582 T ReleaseTools::Feature -- enabled? -- {:feature=>:slack_down}
rake aborted!
2024-09-03 15:20:14.245433 F Rake::Task -- Task failed -- Exception: ReleaseTools::Services::OmnibusPackages::Base::OmnibusPackagesError: No Omnibus packages found for 17.4.0

When all packages were published

bundle exec rake "monthly:verify:check_omnibus_packages_publishing"
2024-09-03 15:21:15.203698 T ReleaseTools::Feature -- enabled? -- {:feature=>:switch_to_main_branch}
2024-09-03 15:21:15.246395 I ReleaseTools::Services::OmnibusPackages::Publishing -- Verifying Omnibus CE/EE package publishing for 17.3.0
2024-09-03 15:21:15.246463 I ReleaseTools::Services::PackagesStatus::Availability -- Fetching pipelines for tag -- {:tag=>"17.3.0+ce.0"}
2024-09-03 15:21:15.671229 D ReleaseTools::GitlabDevClient -- [HTTParty] [2024-09-03 15:21:15 +0200] 200 "GET https://dev.gitlab.org/api/v4/projects/gitlab%2Fomnibus-gitlab/pipelines" - 
2024-09-03 15:21:15.671444 I ReleaseTools::Services::PackagesStatus::Availability -- Fetching pipelines for tag -- {:tag=>"17.3.0+ee.0"}
2024-09-03 15:21:16.074104 D ReleaseTools::GitlabDevClient -- [HTTParty] [2024-09-03 15:21:16 +0200] 200 "GET https://dev.gitlab.org/api/v4/projects/gitlab%2Fomnibus-gitlab/pipelines" - 
2024-09-03 15:21:16.074757 I ReleaseTools::Services::PackagesStatus::Availability -- Fetching bridge -- {:pipeline=>"https://dev.gitlab.org/gitlab/omnibus-gitlab/-/pipelines/340253"}
2024-09-03 15:21:16.606730 D ReleaseTools::GitlabDevClient -- [HTTParty] [2024-09-03 15:21:16 +0200] 200 "GET https://dev.gitlab.org/api/v4/projects/gitlab%2Fomnibus-gitlab/pipelines/340253/bridges" - 
2024-09-03 15:21:17.053206 D ReleaseTools::GitlabDevClient -- [HTTParty] [2024-09-03 15:21:17 +0200] 200 "GET https://dev.gitlab.org/api/v4/projects/gitlab%2Fomnibus-gitlab/pipelines/340481" - 
2024-09-03 15:21:17.053812 I ReleaseTools::Services::PackagesStatus::Availability -- Downstream pipeline retrieved successfully for the tag: 17.3.0+ce.0 -- {:downstream_pipeline=>"https://dev.gitlab.org/gitlab/omnibus-gitlab/-/pipelines/340481", :job_id=>340481, :status=>"success"}
2024-09-03 15:21:17.053854 I ReleaseTools::Services::PackagesStatus::Availability -- Fetching bridge -- {:pipeline=>"https://dev.gitlab.org/gitlab/omnibus-gitlab/-/pipelines/340254"}
2024-09-03 15:21:17.641353 D ReleaseTools::GitlabDevClient -- [HTTParty] [2024-09-03 15:21:17 +0200] 200 "GET https://dev.gitlab.org/api/v4/projects/gitlab%2Fomnibus-gitlab/pipelines/340254/bridges" - 
2024-09-03 15:21:18.094047 D ReleaseTools::GitlabDevClient -- [HTTParty] [2024-09-03 15:21:18 +0200] 200 "GET https://dev.gitlab.org/api/v4/projects/gitlab%2Fomnibus-gitlab/pipelines/340483" - 
2024-09-03 15:21:18.094564 I ReleaseTools::Services::PackagesStatus::Availability -- Downstream pipeline retrieved successfully for the tag: 17.3.0+ee.0 -- {:downstream_pipeline=>"https://dev.gitlab.org/gitlab/omnibus-gitlab/-/pipelines/340483", :job_id=>340483, :status=>"success"}
2024-09-03 15:21:18.094676 I ReleaseTools::Services::OmnibusPackages::Publishing -- Packages for 17.3.0+ce.0, 17.3.0+ee.0 have completed publishing
2024-09-03 15:21:18.094777 I ReleaseTools::Slack::ReleaseJobEndNotifier -- Posting slack message -- {:job_type=>"Verify package publishing", :status=>:success}
2024-09-03 15:21:18.095259 T ReleaseTools::Slack::Message -- post -- {:channel=>"C0139MAV672", :text=>"Verify package publishing job successfully executed - http://test.com/", :unfurl_links=>false, :unfurl_media=>false, :blocks=>[{:type=>"section", :text=>{:type=>"mrkdwn", :text=>":release: :ci_passing: *Verify package publishing job <http://test.com/|successfully> executed*"}}, {:type=>"context", :elements=>[{"type"=>"mrkdwn", "text"=>":clock1: 2024-09-03 13:21 UTC"}]}]}
2024-09-03 15:21:18.095305 T ReleaseTools::Feature -- enabled? -- {:feature=>:slack_down}

Author Check-list

  • Has documentation been updated?
Edited by Dat Tang

Merge request reports