Re-evaluate approval policies after an agentic analyzer scan

Meir Benayounrequested to merge
bl-split/mr4e-policy-reevaluation into master

What does this MR do and why?

  • Approval policies are re-evaluated when a recognized agentic-analyzer workload pipeline (a duo_workflow workload pipeline with a succeeded SAST scan) finishes, not only when the CI pipeline does.
  • Only open MRs whose head is the scanned sha are re-evaluated, each against its own head pipeline, through the existing SyncMergeRequestApprovalsWorker. MRs whose head pipeline is still running are skipped, as in the CI path.
  • Gated by agentic_analyzer_security_ingestion; other pipelines behave as before.
  • Generic to agentic analyzers: nothing here is specific to one flow. Business logic security scanning is the first one.

Part of the BLSA split of !246889. Tracker: https://gitlab.com/gitlab-org/gitlab/-/work_items/630266

Before / after

Situation Before After
Agentic analyzer scan finishes after the CI pipeline (the usual case) Findings are not evaluated until something else re-runs the sync, so they rarely block the merge The MR's approval rules are re-evaluated with the scan's findings
Agentic analyzer scan finishes while the CI pipeline is still running Evaluated when the CI pipeline completes Unchanged in effect: the workload pipeline skips the MR (unless the head pipeline already has security findings) and policies are re-evaluated when the CI pipeline completes, which picks up the scan's findings
Flag off, other pipeline sources, or MRs at other shas No sync from the workload pipeline Unchanged: no sync

How to review

  1. Security::AgenticAnalyzer.scan_pipeline? recognizes the workload pipeline with the same lookup !257196 (merged) uses to add it to policy evaluation (scan_pipeline_ids_for), so the sync only fires when the evaluation will actually see the scan.
  2. Security::AgenticAnalyzer.opened_merge_requests_at finds open MRs whose latest diff head is the scanned sha (driven from merge_request_diffs.head_commit_sha with ANY(ARRAY(...)), newest first by id, limited to 20; see Database queries).
  3. SyncFindingsToApprovalRulesService#sync_scan_finding enqueues SyncMergeRequestApprovalsWorker with each MR's diff_head_pipeline, before the existing CI-source guard. It skips MRs with no head pipeline, and MRs whose head pipeline is not complete_or_manual? and has no security findings (the same guard the CI path applies). It is reached from the existing ProcessPipelineCompletionWorker on Ci::PipelineFinishedEvent / Security::ReportsIngestedEvent.

Backward compatibility

  • Flag-gated by agentic_analyzer_security_ingestion. With it off, scan_pipeline? returns false with no queries and the service behaves exactly as before.
  • No change for CI, security orchestration or any other pipeline: the new branch only matches a recognized agentic-analyzer workload pipeline (a duo_workflow workload pipeline with a succeeded SAST scan).
  • No new worker, schema or evaluation path.

Database queries

Security::AgenticAnalyzer.opened_merge_requests_at finds the open MRs whose latest diff head is the scanned sha. It runs when an agentic analyzer workload pipeline with a succeeded scan completes.

SQL from to_sql on this MR's code (Rails test env, project.id = 278964):

SELECT "merge_requests".* FROM "merge_requests"
WHERE "merge_requests"."target_project_id" = 278964
  AND "merge_requests"."state_id" = 1
  AND (merge_requests.latest_merge_request_diff_id = ANY(ARRAY(
    SELECT "merge_request_diffs"."id" FROM "merge_request_diffs"
    WHERE "merge_request_diffs"."head_commit_sha" = '<scanned sha>'
  )))
ORDER BY "merge_requests"."id" DESC
LIMIT 20

The preloads (head_pipeline, latest_merge_request_diff) are primary-key lookups for the at most 20 MRs returned.

Indexes used

  • index_on_merge_request_diffs_head_commit_sha (head_commit_sha): collects the diff ids at the sha.
  • index_merge_requests_on_latest_merge_request_diff_id (latest_merge_request_diff_id): finds the MRs whose latest diff is one of them. The project and state filters then apply to those few rows.

Why ANY(ARRAY(...)): it collects the diff ids first, so the query always starts from the sha index, never by walking merge_requests by id and checking diffs per row (the slow case when nothing matches). MergeRequest.by_commit_sha is moving to the same form behind mr_by_commit_sha_use_array_subquery.

Timings (Database Lab, gitlab-production-main, project 278964, 2026-10-02, postgresai joe explain). Cold is the first run, warm the run right after. The match sha is the head of an open gitlab-org/gitlab MR (1 row); the no-match sha is made up.

No match Match
Cold 18.6 ms (execution 8.8 ms) 28.6 ms (execution 18.5 ms)
Warm 10.1 ms (execution 0.10 ms) 11.4 ms (execution 0.14 ms)

Planning is about 10 ms in every run; the plan shape is the same in all four. The CLI returned no shareable plan link; Joe command ids 508480, 508482, 508484, 508486.

Plan: match, cold 
 Limit  (cost=51.72..51.73 rows=1 width=1043) (actual time=18.401..18.403 rows=1 loops=1)
   Buffers: shared hit=5 read=9 dirtied=2
   WAL: records=3 fpi=2 bytes=14668
   I/O Timings: read=18.217 write=0.000
   InitPlan 1
     ->  Index Scan using index_on_merge_request_diffs_head_commit_sha on public.merge_request_diffs  (cost=0.70..20.41 rows=12 width=8) (actual time=12.383..12.385 rows=1 loops=1)
           Index Cond: ((merge_request_diffs.head_commit_sha)::text = 'e172ab259c5782fbb1520074181ddaaffea375e7'::text)
           Buffers: shared hit=1 read=5 dirtied=1
           WAL: records=1 fpi=1 bytes=7601
           I/O Timings: read=12.306 write=0.000
   ->  Sort  (cost=31.31..31.32 rows=1 width=1043) (actual time=18.399..18.400 rows=1 loops=1)
         Sort Key: merge_requests.id DESC
         Sort Method: quicksort  Memory: 25kB
         Buffers: shared hit=5 read=9 dirtied=2
         WAL: records=3 fpi=2 bytes=14668
         I/O Timings: read=18.217 write=0.000
         ->  Index Scan using index_merge_requests_on_latest_merge_request_diff_id on public.merge_requests  (cost=0.57..31.30 rows=1 width=1043) (actual time=18.379..18.380 rows=1 loops=1)
               Index Cond: (merge_requests.latest_merge_request_diff_id = ANY ((InitPlan 1).col1))
               Filter: ((merge_requests.target_project_id = 278964) AND (merge_requests.state_id = 1))
               Buffers: shared hit=2 read=9 dirtied=2
               WAL: records=3 fpi=2 bytes=14668
               I/O Timings: read=18.217 write=0.000
Plan: no match, cold 
 Limit  (cost=51.72..51.73 rows=1 width=1043) (actual time=8.741..8.743 rows=0 loops=1)
   Buffers: shared hit=4 read=4
   I/O Timings: read=8.660 write=0.000
   InitPlan 1
     ->  Index Scan using index_on_merge_request_diffs_head_commit_sha on public.merge_request_diffs  (cost=0.70..20.41 rows=12 width=8) (actual time=8.715..8.715 rows=0 loops=1)
           Index Cond: ((merge_request_diffs.head_commit_sha)::text = 'b34603c94075c20dd4d3a1a61b9b42294495a426'::text)
           Buffers: shared hit=1 read=4
           I/O Timings: read=8.660 write=0.000
   ->  Sort  (cost=31.31..31.32 rows=1 width=1043) (actual time=8.740..8.741 rows=0 loops=1)
         Sort Key: merge_requests.id DESC
         Sort Method: quicksort  Memory: 25kB
         Buffers: shared hit=4 read=4
         I/O Timings: read=8.660 write=0.000
         ->  Index Scan using index_merge_requests_on_latest_merge_request_diff_id on public.merge_requests  (cost=0.57..31.30 rows=1 width=1043) (actual time=8.726..8.726 rows=0 loops=1)
               Index Cond: (merge_requests.latest_merge_request_diff_id = ANY ((InitPlan 1).col1))
               Filter: ((merge_requests.target_project_id = 278964) AND (merge_requests.state_id = 1))
               Buffers: shared hit=1 read=4
               I/O Timings: read=8.660 write=0.000

Design decisions

  • Reuse the MR's head pipeline. The existing worker and UpdateApprovalsService do the evaluation; !257196 (merged) already pulls the workload pipeline's findings in through RelatedPipelines. No new evaluation path.
  • Hook into the existing completion path (SyncFindingsToApprovalRulesService) instead of the Ci::Pipeline state machine, so it runs after the scans are stored and inherits the policy-availability and license checks.
  • Recognition requires a succeeded scan, which matches what the evaluation includes and makes the trigger a no-op for workload pipelines without one.
  • Idempotent and cheap: the sync worker is idempotent and deduplicated; the MR lookup is bounded (20) and restricted to opened MRs.

Known limitations

  • An MR with no head pipeline at the scanned sha is skipped. It is evaluated when its CI pipeline completes.
  • More than 20 open MRs at one sha: only the 20 newest (highest id) are re-evaluated.
  • Forked merge requests: the lookup uses the workload pipeline's project as the target, so fork MRs aren't re-evaluated from this path. Business logic scans don't run for fork MRs today; any future analyzer that does relies on the CI-completion sync.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Meir Benayoun

Merge request reports