Notify security implementation issues when they are not qualified for a security release (!2637) · Merge requests · GitLab.org / release-tools

Steve Abrams requested to merge delivery19570-notify-unqualified into master Sep 14, 2023

🔍 What does this MR do and why?

In gitlab-com/gl-infra&1061 (closed), we are working to automate and improve the early merge process.

In gitlab-com/gl-infra/delivery#19568 (closed) we added a service that will automatically link and unlink qualified security implementation issues to the current security release tracking issue.

In this issue, when a security implementation issue is processed, but is not ready and also not yet linked, we:

Post a comment to the security implementation issue with a list of reasons why it is not ready
Remove the security-target label from the security implementation issue

We also update the existing notification for when an issue is unlinked to include the list of reasons.

The listed reasons will be expanded to include more detail and helpful instructions for engineers in a future issue/MR.

In order to include a full list, Security::ImplementationIssue was updated to not have a single pending_reason, but have multiple pending_reasons. The reason for this is that we don't want to ping an engineer saying they are missing merge requests, then they add the last one only to get pinged again saying they have unassigned merge requests. So with this update I've also updated the builder that builds the table for the security release tracking issue so it also lists a comma separated list of all the reasons.

💻 Testing

To test, I used two fake issues:

https://gitlab.com/gitlab-org/security/gitlab/-/issues/971 - fake security implementation issue
https://gitlab.com/gitlab-org/gitlab/-/issues/424529 - fake security release tracking issue

I made the following modifications locally to force the use of those issues and also force the fake implementation issue to have some problems:

git diff

diff --git a/lib/release_tools/security/implementation_issue.rb b/lib/release_tools/security/implementation_issue.rb
index 5eb96b63..0ec04cc1 100644
--- a/lib/release_tools/security/implementation_issue.rb
+++ b/lib/release_tools/security/implementation_issue.rb
@@ -56,9 +56,11 @@ module ReleaseTools
       end

       def ready_to_be_processed?
-        reject('missing merge requests') if missing_backports?
+        return unless iid == 971
+
+        reject('missing merge requests')
         reject('invalid merge requests status') unless merge_requests_with_allowed_status?
-        reject('unassigned merge requests') unless merge_requests_assigned_to_the_bot?
+        reject('unassigned merge requests')
         reject('invalid merge requests') unless valid_merge_requests?

         return false if @pending_reasons.any?
diff --git a/lib/release_tools/security/issue_crawler.rb b/lib/release_tools/security/issue_crawler.rb
index 520d2fda..5245f676 100644
--- a/lib/release_tools/security/issue_crawler.rb
+++ b/lib/release_tools/security/issue_crawler.rb
@@ -12,7 +12,7 @@ module ReleaseTools
       SECURITY_NAMESPACE = 'gitlab-org/security'

       # The label required by security root/meta issues.
-      ROOT_ISSUE_LABEL = 'upcoming security release'
+      ROOT_ISSUE_LABEL = 'security-target'

       # The label required for a related issue or merge request to be considered
       # by the crawler.
@@ -96,7 +96,7 @@ module ReleaseTools
       # Returns issues that are related to the security release tracking issue.
       def related_security_issues
         security_issues_and_merge_requests_for(
-          security_issues_for(release_issue.iid)
+          security_issues_for(424529)
         )
       end

I then ran the issue processor locally:

~/workspace/gitlab-org/release-tools (delivery19570-notify-unqualified ✗) RELEASE_BOT_PRODUCTION_TOKEN=REDACTED be pry --gem
[1] pry(main)> ReleaseTools::Security::TargetIssuesProcessor.new.execute
2023-09-14 15:19:07.331564 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:19:07 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fcharts%2Fcomponents%2Fimages/issues" 2
2023-09-14 15:19:07.591283 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:19:07 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitaly/issues" 2
2023-09-14 15:19:07.984286 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:19:07 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/issues" -
2023-09-14 15:19:08.298862 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:19:08 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab-pages/issues" 2
2023-09-14 15:19:08.618458 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:19:08 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fomnibus-gitlab/issues" 2
2023-09-14 15:19:09.549577 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:19:09 -0600] 200 "GET https://gitlab.com/api/v4/projects/15642544/issues/971/related_merge_requests" -
2023-09-14 15:19:09.880427 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:19:09 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/issues" -
2023-09-14 15:19:09.881485 W Ruby -- WARNING: Please convert ObjectifiedHash object to hash before calling Hash methods on it.
 -- {:source=>"ruby_warnings", :stacktrace=>"/Users/steveabrams/workspace/gitlab-org/release-tools/lib/release_tools/issuable.rb:6:in `initialize'"}
2023-09-14 15:19:09.882761 I ReleaseTools::Security::TargetIssuesProcessor -- 1 target issues found. They will be evaluated and considered for linking to the security release tracking issue: https://gitlab.com/gitlab-org/gitlab/-/issues/424529.
2023-09-14 15:19:09.882854 W ReleaseTools::Security::ImplementationIssue -- Rejecting implementation issue due to missing merge requests -- {:url=>"https://gitlab.com/gitlab-org/security/gitlab/-/issues/971"}
2023-09-14 15:19:09.882889 W ReleaseTools::Security::ImplementationIssue -- Rejecting implementation issue due to invalid merge requests status -- {:url=>"https://gitlab.com/gitlab-org/security/gitlab/-/issues/971"}
2023-09-14 15:19:09.882897 W ReleaseTools::Security::ImplementationIssue -- Rejecting implementation issue due to unassigned merge requests -- {:url=>"https://gitlab.com/gitlab-org/security/gitlab/-/issues/971"}
2023-09-14 15:19:10.201368 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:19:10 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/issues/424529/links" -
2023-09-14 15:19:10.201963 I ReleaseTools::Security::TargetIssuesProcessor -- https://gitlab.com/gitlab-org/security/gitlab/-/issues/971 is not ready to be processed or linked to the security release tracking issue.
2023-09-14 15:19:10.696744 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:19:10 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/issues" -
2023-09-14 15:19:11.532287 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:19:11 -0600] 201 "POST https://gitlab.com/api/v4/projects/15642544/issues/971/notes" 950
=> [#<ReleaseTools::Security::ImplementationIssue:0x000000010b3dcf50
  @iid=971,
  @issue=
   #<Gitlab::ObjectifiedHash:217140 {hash: {"id"=>133377123, "iid"=>971, ......},
  @merge_request_targeting_default_branch=nil,
  @merge_requests=[],
  @pending_reasons=["missing merge requests", "invalid merge requests status", "unassigned merge requests"],
  @project_id=15642544,
  @reference="gitlab-org/security/gitlab#971",
  @semantic_logger=#<SemanticLogger::Logger:0x000000010b3d7370 @filter=nil, @level=nil, @level_index=nil, @name="ReleaseTools::Security::ImplementationIssue">,
  @web_url="https://gitlab.com/gitlab-org/security/gitlab/-/issues/971">]

We see the comment successfully posted and the security-target label was removed:

Next, I re-added the security-target label, linked the implementation issue to the security release tracking issue, and ran the processor again:

[3] pry(main)> ReleaseTools::Security::TargetIssuesProcessor.new.execute
2023-09-14 15:23:17.236031 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:23:17 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fcharts%2Fcomponents%2Fimages/issues" 2
2023-09-14 15:23:17.488993 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:23:17 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitaly/issues" 2
2023-09-14 15:23:17.882902 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:23:17 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/issues" -
2023-09-14 15:23:18.190482 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:23:18 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab-pages/issues" 2
2023-09-14 15:23:18.474428 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:23:18 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fomnibus-gitlab/issues" 2
2023-09-14 15:23:18.922288 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:23:18 -0600] 200 "GET https://gitlab.com/api/v4/projects/15642544/issues/971/related_merge_requests" -
2023-09-14 15:23:19.255881 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:23:19 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/issues" -
2023-09-14 15:23:19.256391 W Ruby -- WARNING: Please convert ObjectifiedHash object to hash before calling Hash methods on it.
 -- {:source=>"ruby_warnings", :stacktrace=>"/Users/steveabrams/workspace/gitlab-org/release-tools/lib/release_tools/issuable.rb:6:in `initialize'"}
2023-09-14 15:23:19.256750 I ReleaseTools::Security::TargetIssuesProcessor -- 1 target issues found. They will be evaluated and considered for linking to the security release tracking issue: https://gitlab.com/gitlab-org/gitlab/-/issues/424529.
2023-09-14 15:23:19.256771 W ReleaseTools::Security::ImplementationIssue -- Rejecting implementation issue due to missing merge requests -- {:url=>"https://gitlab.com/gitlab-org/security/gitlab/-/issues/971"}
2023-09-14 15:23:19.256785 W ReleaseTools::Security::ImplementationIssue -- Rejecting implementation issue due to invalid merge requests status -- {:url=>"https://gitlab.com/gitlab-org/security/gitlab/-/issues/971"}
2023-09-14 15:23:19.256791 W ReleaseTools::Security::ImplementationIssue -- Rejecting implementation issue due to unassigned merge requests -- {:url=>"https://gitlab.com/gitlab-org/security/gitlab/-/issues/971"}
2023-09-14 15:23:19.721473 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:23:19 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/issues/424529/links" -
2023-09-14 15:23:20.231481 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:23:20 -0600] 200 "GET https://gitlab.com/api/v4/projects/15642544/issues/971/related_merge_requests" -
2023-09-14 15:23:20.232509 I ReleaseTools::Security::TargetIssuesProcessor -- https://gitlab.com/gitlab-org/security/gitlab/-/issues/971 will be unlinked from  the security release tracking issue as it is no longer ready to be processed.
2023-09-14 15:23:20.721556 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:23:20 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/issues" -
2023-09-14 15:23:21.029941 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:23:21 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/issues/424529/links" -
2023-09-14 15:23:21.626190 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:23:21 -0600] 200 "DELETE https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/issues/424529/links/4289857" -
2023-09-14 15:23:22.006600 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:23:22 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/issues" -
2023-09-14 15:23:22.522917 D ReleaseTools::GitlabClient -- [HTTParty] [2023-09-14 15:23:22 -0600] 201 "POST https://gitlab.com/api/v4/projects/15642544/issues/971/notes" 977
=> [#<ReleaseTools::Security::ImplementationIssue:0x000000010b338360
  @iid=971,
  @issue=
   #<Gitlab::ObjectifiedHash:217200 {hash: {"id"=>133377123, "iid"=>971,..........},
  @merge_request_targeting_default_branch=nil,
  @merge_requests=[],
  @pending_reasons=["missing merge requests", "invalid merge requests status", "unassigned merge requests"],
  @project_id=15642544,
  @reference="gitlab-org/security/gitlab#971",
  @semantic_logger=#<SemanticLogger::Logger:0x000000010b3d7370 @filter=nil, @level=nil, @level_index=nil, @name="ReleaseTools::Security::ImplementationIssue">,
  @web_url="https://gitlab.com/gitlab-org/security/gitlab/-/issues/971">]

We see the label was removed again, the issue was unlinked, and the updated comment was posted to the issue:

Author Check-list

[-] Has documentation been updated?

Edited Sep 14, 2023 by Steve Abrams

Notify security implementation issues when they are not qualified for a security release

🔍 What does this MR do and why?

💻 Testing

Author Check-list

Merge request reports