Create new worker for syncing approval rules from security_findings

Why are we doing this work

Currently we are performing all calculations based on security reports assigned to the pipeline. This means we have to parse data from Security Report JSON files and perform all calculations in Ruby. It is a part of the code that was related to old Vulnerability-Check feature and during the development on Scan Result Policies it was not changed, but the way to configure policies.

As more and more customers are using Scan Result Policies and we want to make sure this feature performs well on larger scale we need to change the way we are currently performing this calculations and move them to service that is more performant to do them - to database.

We already have Security::Scan and Security::Finding models and data in database created for all pipelines, so we could reuse it. We have to make sure we have this data when ::Ci::SyncReportsToReportApprovalRulesWorker is called (to perform checks on Approval Rules), as at the same time ::Security::StoreScansWorker is called at the same time (to store Security::Finding and Security::Scan data in database).

Relevant links

Non-functional requirements

  • Documentation: No change is needed
  • Feature flag: Hide the changes behind feature flag sync_approval_rules_from_findings
  • Performance: perform performance benchmarking to see how it is different from the current method
  • Testing: cover with unit tests with multiple scenarios (ie. checking this with detached pipeline, etc.)

Implementation Plan

  • backend Create a new feature flag sync_approval_rules_from_findings
  • backend Update Ci::SyncReportsToApprovalRulesService to check the feature flag before calling sync_scan_finding
  • backend Create a new service Security::SyncFindingsToApprovalRulesService which syncs the approval rules from security_findings for the given pipeline
  • backend Create a new worker Security::SyncFindingsToApprovalRulesWorker that calls Security::SyncFindingsToApprovalRulesService
  • backend Update Security::StoreScansService to invoke the Security::SyncFindingsToApprovalRulesWorker after the findings are persisted and check the feature flag

Verification steps

  • Create a project and configure a scan result policy for the project to require approvals when vulnerabilities are detected.
  • Enable the feature flag for the project
  • Create a MR in the project with multiple security scan enabled containing a huge amount of vulnerabilities
  • Verify if the MR approvals are applied and existing features (enforce_scan_result_policies_for_preexisting_vulnerabilities and require approval on scan removal) works fine
Edited by Sashi Kumar Kumaresan