Create new worker for syncing approval rules from security_findings
Why are we doing this work
Currently we are performing all calculations based on security reports assigned to the pipeline. This means we have to parse data from Security Report JSON files and perform all calculations in Ruby. It is a part of the code that was related to old Vulnerability-Check
feature and during the development on Scan Result Policies it was not changed, but the way to configure policies.
As more and more customers are using Scan Result Policies and we want to make sure this feature performs well on larger scale we need to change the way we are currently performing this calculations and move them to service that is more performant to do them - to database.
We already have Security::Scan
and Security::Finding
models and data in database created for all pipelines, so we could reuse it. We have to make sure we have this data when ::Ci::SyncReportsToReportApprovalRulesWorker
is called (to perform checks on Approval Rules), as at the same time ::Security::StoreScansWorker
is called at the same time (to store Security::Finding
and Security::Scan
data in database).
Relevant links
Non-functional requirements
-
Documentation: No change is needed -
Feature flag: Hide the changes behind feature flag sync_approval_rules_from_findings
-
Performance: perform performance benchmarking to see how it is different from the current method -
Testing: cover with unit tests with multiple scenarios (ie. checking this with detached pipeline, etc.)
Implementation Plan
-
backend Create a new feature flag sync_approval_rules_from_findings
-
backend Update Ci::SyncReportsToApprovalRulesService
to check the feature flag before callingsync_scan_finding
-
backend Create a new service Security::SyncFindingsToApprovalRulesService
which syncs the approval rules fromsecurity_findings
for the given pipeline -
backend Create a new worker Security::SyncFindingsToApprovalRulesWorker
that callsSecurity::SyncFindingsToApprovalRulesService
-
backend Update Security::StoreScansService
to invoke theSecurity::SyncFindingsToApprovalRulesWorker
after the findings are persisted and check the feature flag
Verification steps
-
Create a project and configure a scan result policy for the project to require approvals when vulnerabilities are detected. -
Enable the feature flag for the project -
Create a MR in the project with multiple security scan enabled containing a huge amount of vulnerabilities -
Verify if the MR approvals are applied and existing features ( enforce_scan_result_policies_for_preexisting_vulnerabilities
and require approval on scan removal) works fine