Audit policy violations only when they change
What does this MR do and why?
Every MR approval policy evaluation that leaves violations on a merge request enqueues MergeRequests::PolicyViolationsDetectedAuditEventWorker and Security::ScanResultPolicies::AuditWarnModeMergeRequestApprovalSettingsWorker. Nothing checks whether the violations changed, so an MR that is re-evaluated many times with the same findings emits the same audit events each time. On GitLab.com over 7 days, the violation comment and audit workers ran about 53M jobs and used about 31% of the policy category's Sidekiq DB time. Within 24 hours the limiter deferred these jobs 2.2M times.
This MR makes UpdateViolationsService take a snapshot of the merge request's violations before and after it writes them. It enqueues the two audit workers only when the snapshot differs and the existing conditions still hold (violations exist and none are running).
The snapshot holds, for each violation: scan_result_policy_id, approval_policy_rule_id and violation_data. A new or removed policy, a new finding, a different license or a different error all count as a change, so they still emit. Two normalizations stop identical re-evaluations from looking changed:
violation_data.contextis excluded. It only holds the evaluatedpipeline_idsandtarget_pipeline_ids, which change on every new pipeline even when the violation is the same.- Arrays are sorted. The order of finding UUIDs, license names, commits and missing scans carries no meaning.
Not changed:
MergeRequests::ViolationsUpdatedEventstill publishes on every evaluation. Its only subscriber,MergeRequests::ProcessAutoMergeFromEventWorker, drives auto-merge, so it is not a pure side effect.- The resolved audit event is unchanged. It fires only when rows with data are actually deleted and none remain, so it already fires once per transition.
The change is behind the audit_policy_violations_on_change_only feature flag (gitlab_com_derisk, disabled by default). With the flag off, behaviour is unchanged.
Audit event semantics change (needs PM sign-off)
With the flag on, security_policy_violations_detected and policy_warn_mode_merge_request_approval_settings_overrides are recorded once per change in the violations, not once per evaluation. Audit streaming destinations get fewer, non-duplicate events. A consumer that counts these events as "number of evaluations" would see lower numbers. A change in any violation data counts as a change. A policy switched between enforced and warn with unchanged violations is recorded on the next change in its violations, not at the switch. The PM decision is still open, so this needs PM sign-off before the flag is rolled out.
How to set up and validate locally
- Enable the flag:
Feature.enable(:audit_policy_violations_on_change_only). - Create a project with an MR approval policy that requires approval for any new critical finding, and open an MR whose pipeline reports such a finding.
- Re-run the MR pipeline without changing the code. With the flag on, the second evaluation does not enqueue
MergeRequests::PolicyViolationsDetectedAuditEventWorker(checklog/sidekiq.logor the audit events page for a singlesecurity_policy_violations_detectedevent). - Push a commit that adds another finding. The workers are enqueued again.
- Disable the flag and repeat step 3: an event is recorded on every evaluation, as before.
Specs: ee/spec/services/security/security_orchestration_policies/update_violations_service_spec.rb.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.