Merge agentic analyzer scan results with SAST in MR security views

Meir Benayounrequested to merge
bl-split/mr4f-merge-with-sast-results into master

What does this MR do and why?

Problem: today an agentic analyzer scan either replaces the SAST results in the MR widget or is invisible.

Generic to agentic analyzers: nothing here is specific to one flow. Business logic security scanning is the first one.

  • One comparison, both scanners. The MR security widget now compares the agentic analyzer scan's SAST findings together with the regular SAST findings (Semgrep, GitLab Advanced SAST), instead of showing one or the other.
  • Like-for-like base. Agentic analyzer findings are compared against the latest agentic analyzer scan of the target branch, so findings that already exist there no longer look "new", and regular SAST findings no longer look "fixed".

Stacked on !257196 (merged) (MR-4, the interim either/or design this replaces). Everything stays behind agentic_analyzer_security_ingestion.

Before / after

Before (MR-4) After
MR widget head The agentic analyzer workload pipeline replaced the head pipeline only when the head (or its child pipelines) had no successful SAST scan. Any successful regular SAST scan hid every agentic analyzer finding. The latest full and latest partial agentic analyzer scan pipelines at the head SHA are added to the head side of the comparison (the related-pipelines support in Vulnerabilities::CompareSecurityReportsService). Both scanners' findings show.
MR widget base Latest target-branch pipeline with any security report. An agentic analyzer head compared against a Semgrep/GitLab Advanced SAST base: all agentic analyzer findings "new", all base SAST findings "fixed". Regular base plus the agentic analyzer base of the target branch, only when the head has an agentic analyzer scan too. When the agentic analyzer scan is the only head, it is compared against the agentic analyzer base alone, so other scanners' findings are not reported as fixed.
Agentic analyzer base lookup Exact match on the target branch HEAD SHA. Scans run on a schedule, so the HEAD has usually moved on: no base, every pre-existing finding "new". Latest full (not partial) agentic analyzer scan of a commit on the target branch (a Gitaly ancestry check, cached).

Preflight fixes in this revision:

  • No agentic analyzer base without an agentic analyzer head. With a regular head and a missing, running or failed agentic analyzer scan, the full-scan tab used to report every agentic analyzer finding on the target branch as "fixed". The base is now only added (and only used as a fallback base) when the head has an agentic analyzer scan.
  • Re-runs at one commit are not merged. Only the latest full run and the latest partial run at a commit are used, so an older run's findings do not stay "added". This now comes from scan_pipeline_ids_for in !257196 (merged).

How to review

  1. ee/app/models/ee/merge_request.rb: comparison_base_pipeline, security_comparison_head_pipeline, the new security_comparison_additional_pipeline_ids, and has_sast_reports?. This is where the head/base pipeline set is decided.
  2. ee/app/services/security/agentic_analyzer.rb: latest_branch_scan_pipeline_for (the base lookup). recent_ids_for_sha_and_source now takes a nil sha (branch lookup).
  3. ee/app/services/vulnerabilities/compare_security_reports_service.rb: the extra head/base pipeline ids join related_pipeline_ids_for, the comparison cache key, and the partial-scan scanner filter.
  4. ee/app/services/security/merge_request_security_report_generation_service.rb: passes the extra ids to the comparison, not the reactive cache params.
  5. Specs: ee/spec/services/security/agentic_analyzer_mr_widget_e2e_spec.rb runs the real comparison service against real scans and findings for each case; the reactive cache is stubbed to compute inline.

Backward compatibility / impact

Consumer Change with the flag on Risk Mitigation
MR widget security comparison (SAST) Head and base include the agentic analyzer scan pipelines Wrong "new"/"fixed" if the pipeline set is wrong Guarded to heads that have an agentic analyzer scan; e2e specs for each head/base combination
has_sast_reports? callers (MR page sast_report_available, rapid diffs presenter) Also true when the latest agentic analyzer scan at the head SHA has a SAST report, even with a regular head pipeline SAST UI shown for more MRs Only with a real SAST report on that scan
  • Flag off: no change. security_comparison_additional_pipeline_ids returns {}, the comparison params and cache key are identical, head is diff_head_pipeline, base is latest_scan_finding_comparison_pipeline. Specs cover the flag-off widget.
  • Non-SAST report types: no change.
  • Cache: the extra ids are part of the comparison key, not the reactive cache identity. A new agentic analyzer scan invalidates the cached report once, like a new related pipeline already does.
  • Request-path cost (flag on), per comparison key: for the target branch base, one capped CI pipelines lookup (the newest 5,000 pipelines in the recent partitions, up to 500 duo_workflow ones), one security_scans query for their full SAST scans (up to 50), three small id lookups to keep the BLSA flow ones, up to 5 pipeline loads and up to 5 cached Gitaly ancestry checks: about 12 queries at most, 3 when no full scan is in the window. Plus one 50-row CI pipelines lookup at the head SHA.
  • Trade-off: if more than 50 newer full SAST scans in the window came from other flows, an older BLSA base is missed and agentic analyzer findings show as new.

Design decisions / Known limitations

  • Why not extend Security::RelatedPipelinesFinder with duo_workflow: it keeps the latest pipeline per source, so a newer non-business-logic Duo pipeline would hide the scan; it sits behind security_report_related_pipelines; and it only covers the head side.
  • Combining is safe per finding. A finding UUID covers the report type, primary identifier and location. Other SAST analyzers use their own rule IDs as the primary identifier, so the two scanners' findings stay apart. A true duplicate is kept once.
  • Agentic analyzer base lookup is bounded. It reads the newest 5,000 pipelines of the project in the recent partitions, keeps up to 500 duo_workflow pipelines from the BLSA flow, then checks up to 5 full scans for being on the target branch. If the base scan falls outside that window, the base is nil and agentic analyzer findings show as new. A partial index would lift the cap (follow-up).
  • No dispatch-branch filter on this base. MR-4c (!257359 (merged)) records the dispatch branch. Here "a full scan of a commit on the target branch" stands in for it, and partial (MR-scoped) scans are excluded.
  • Head uses diff_head_sha, like the other MR-4 call sites. MR-4d's scanned_sha is not on this base.
  • A regular head without SAST, against a base with regular SAST, still reports the base's regular SAST findings as fixed. That is existing behaviour for any scanner missing from the head.
  • Policies are out of scope. Merge request approval policy evaluation is handled in the policy re-evaluation MR.

Acceptance checklist

  • MR widget with both regular SAST and agentic analyzer findings shows both (agentic_analyzer_mr_widget_e2e_spec.rb)
  • Agentic analyzer head against an agentic analyzer base does not report regular SAST base findings as fixed, and only new findings are added (same spec)
  • Regular head without an agentic analyzer scan does not report the target branch's agentic analyzer findings as fixed (same spec, agentic_analyzer_merge_request_spec.rb)
  • Regular + agentic analyzer base: a finding on both sides is not new, one only on the base is fixed (e2e spec)
  • Partial agentic analyzer scan on top of a regular head does not report its base findings as fixed in the full-scan tab (e2e spec)
  • Two full runs at one commit: only the latest is used (agentic_analyzer_spec.rb)
  • Agentic analyzer base is found when the target branch moved on since the last scan (agentic_analyzer_spec.rb, agentic_analyzer_merge_request_spec.rb)
  • Flag off leaves the widget unchanged (e2e spec)
  • CI green

🤖 Generated with Claude Code

Database queries

All SQL below is from to_sql on this MR's code (Rails test env), with gitlab.com values substituted: project 278964 (gitlab-org/gitlab), source = 17 (duo_workflow), scan_type = 1 (sast), recent CI partitions 114, 115, 116, and real gitlab-org/gitlab pipeline ids. Timings: Database Lab, postgresai joe explain, 2026-10-03. Cold is the first run, warm the run right after.

Base lookup: Security::AgenticAnalyzer.latest_branch_scan_pipeline_for

Runs when an MR's head has an agentic analyzer scan, at most once per report request (memoized). A bounded walk: newest 5,000 project pipelines → up to 500 duo_workflow ones → one security_scans query → the flow filter → up to 5 Gitaly ancestry checks.

1. Recent duo_workflow pipeline ids (Ci::Pipeline.recent_source_ids_in_window, CI database)

SELECT p_ci_pipelines.id FROM (
  SELECT p_ci_pipelines.id, p_ci_pipelines.source FROM p_ci_pipelines
  WHERE p_ci_pipelines.project_id = 278964 AND p_ci_pipelines.partition_id IN (114, 115, 116)
  ORDER BY p_ci_pipelines.id DESC LIMIT 5000
) p_ci_pipelines
WHERE p_ci_pipelines.source = 17
ORDER BY p_ci_pipelines.id DESC LIMIT 500

Uses p_ci_pipelines_project_id_id_idx (project_id, id DESC) per partition, merged by id. There is no (project_id, source, id) index, so the inner limit bounds the read: on gitlab-org/gitlab it reached 500 duo_workflow pipelines after 3,060 rows.

2. Full SAST scans of those pipelines (security database)

SELECT security_scans.* FROM security_scans
LEFT OUTER JOIN vulnerability_partial_scans partial_scan ON partial_scan.scan_id = security_scans.id
WHERE security_scans.project_id = 278964 AND security_scans.scan_type = 1
  AND security_scans.status = 1 AND security_scans.latest = TRUE
  AND security_scans.pipeline_id IN (<the 500 ids from 1>)
  AND partial_scan.scan_id IS NULL
ORDER BY security_scans.pipeline_id DESC LIMIT 50

Uses index_for_security_scans_scan_type (scan_type, project_id, pipeline_id WHERE status = 1) and index_vulnerability_partial_scans_on_scan_id.

3–5. Business logic security flow filter (bl_security_flow_pipeline_ids, at most 50 ids each, one query per database hop)

-- CI database
SELECT p_ci_workloads.id, p_ci_workloads.pipeline_id FROM p_ci_workloads
WHERE p_ci_workloads.partition_id IN (114, 115, 116) AND p_ci_workloads.pipeline_id IN (<≤50 ids>) LIMIT 50
-- main database
SELECT duo_workflows_workloads.workload_id, duo_workflows_workloads.workflow_id FROM duo_workflows_workloads
WHERE duo_workflows_workloads.workload_id IN (<≤50 ids>) LIMIT 50
SELECT duo_workflows_workflows.id FROM duo_workflows_workflows
WHERE duo_workflows_workflows.id IN (<≤50 ids>) AND duo_workflows_workflows.workflow_definition = 'bl_security/experimental' LIMIT 50

Indexes: p_ci_workloads_pipeline_id_idx (pipeline_id, partition_id, unique), index_duo_workflows_workloads_on_workload_id, duo_workflows_workflows_pkey.

Head scans: Security::AgenticAnalyzer.scan_pipeline_ids_for (existing queries, new caller)

MergeRequest#agentic_security_head_pipeline_ids now calls this for SAST comparisons when the flag is on: the existing per-sha duo_workflow pipeline lookup (p_ci_pipelines_project_id_sha_idx), then two pick(:pipeline_id) queries on security_scans with the same shape as query 2. SQL unchanged by this MR; per-sha lookup timed below as 6.

Partial-scan scanners: CompareSecurityReportsService (security database)

Only with scan_mode: full and additional head pipelines, so at most 3 pipeline ids (the head pipeline plus up to 2 agentic analyzer scan pipelines).

-- 7. additional_head_partial_scanner_ids
SELECT security_scans.* FROM security_scans
INNER JOIN vulnerability_partial_scans ON vulnerability_partial_scans.scan_id = security_scans.id
WHERE security_scans.pipeline_id IN (<≤2 ids>) AND security_scans.scan_type = 1 AND security_scans.latest = TRUE
-- 8. head_full_scanner_external_ids
SELECT security_scans.* FROM security_scans
LEFT OUTER JOIN vulnerability_partial_scans partial_scan ON partial_scan.scan_id = security_scans.id
WHERE security_scans.pipeline_id IN (<≤3 ids>) AND security_scans.scan_type = 1
  AND security_scans.latest = TRUE AND security_scans.status = 1 AND partial_scan.scan_id IS NULL

Both use index_security_scans_on_pipeline_id_and_scan_type and index_vulnerability_partial_scans_on_scan_id.

Timings

Query Database Cold Warm Buffers (cold) Rows
1. recent duo_workflow ids ci 986 ms (exec 980 ms, I/O 910 ms) 11.0 ms (exec 5.4 ms) hit 336, read 2,867 500
2. full SAST scans sec 10.6 ms (exec 6.1 ms) 3.9 ms (exec 0.21 ms) hit 15, read 7 0
3. workloads ci 27.7 ms (exec 25.9 ms) 2.1 ms (exec 0.20 ms) hit 22, read 45 50
4. workflow workloads main 2.7 ms (exec 2.0 ms) 0.7 ms (exec 0.04 ms) read 3 0
5. BL security workflows main 5.0 ms (exec 3.5 ms) 1.7 ms (exec 0.04 ms) read 3 0
6. per-sha duo_workflow ids (existing) ci 11.3 ms (exec 5.3 ms) 6.2 ms (exec 0.13 ms) hit 6, read 11 3
7. additional partial scans sec 9.4 ms (exec 6.0 ms) 3.8 ms (exec 0.13 ms) hit 6, read 5 0
8. head full scans sec 7.2 ms (exec 3.1 ms) 4.0 ms (exec 0.14 ms) hit 15, read 4 1

Query 1 is the only one with notable cold cost: about one heap page per walked row (it reads source from the heap), so its worst case is about 5,000 page reads, when duo_workflow pipelines are sparse. It runs at most once per MR report request and is warm after that. A partial index on (project_id, id DESC) WHERE source = 17 would remove the walk; not added here.

Inputs: 1 and 2 use the 500 newest real duo_workflow pipeline ids of gitlab-org/gitlab (none has a SAST scan yet, so 2 returns 0 rows and 3–5 would not run); 3 uses 50 real duo_workflow pipeline ids; 4 and 5 use made-up ids (real workload/workflow ids are not exposed by the API); 8 uses a real merge request pipeline plus 2 duo_workflow pipelines. The CLI returns no shareable plan link; Joe command ids: cold 508824, 508828, 508834, 508838, 508840, 508842, 508844, 508846; warm 508848–508862 (even ids).

Plan: query 1, cold 
 Limit  (cost=1.59..7492.03 rows=4 width=8) (actual time=6.919..979.926 rows=500 loops=1)
   Buffers: shared hit=336 read=2867 dirtied=1291
   I/O Timings: read=909.660 write=0.000
   ->  Subquery Scan on p_ci_pipelines  (cost=1.59..7492.03 rows=4 width=8) (actual time=6.918..979.842 rows=500 loops=1)
         Filter: (p_ci_pipelines.source = 17)
         Rows Removed by Filter: 2560
         Buffers: shared hit=336 read=2867 dirtied=1291
         I/O Timings: read=909.660 write=0.000
         ->  Limit  (cost=1.59..7429.53 rows=5000 width=12) (actual time=6.425..979.328 rows=3060 loops=1)
               Buffers: shared hit=336 read=2867 dirtied=1291
               I/O Timings: read=909.660 write=0.000
               ->  Merge Append  (cost=1.59..626530.95 rows=421738 width=12) (actual time=6.424..978.905 rows=3060 loops=1)
                     Sort Key: p_ci_pipelines_1.id DESC
                     Buffers: shared hit=336 read=2867 dirtied=1291
                     I/O Timings: read=909.660 write=0.000
                     ->  Index Scan using ci_pipelines_114_project_id_id_idx on gitlab_partitions_dynamic.ci_pipelines_114 p_ci_pipelines_2  (cost=0.57..293290.72 rows=199274 width=12) (actual time=2.527..2.527 rows=1 loops=1)
                           Index Cond: (p_ci_pipelines_2.project_id = 278964)
                           Filter: (p_ci_pipelines_2.partition_id = ANY ('{114,115,116}'::bigint[]))
                           Buffers: shared read=5
                           I/O Timings: read=2.456 write=0.000
                     ->  Index Scan using ci_pipelines_115_project_id_id_idx on gitlab_partitions_dynamic.ci_pipelines_115 p_ci_pipelines_3  (cost=0.57..325875.02 rows=221195 width=12) (actual time=3.510..725.452 rows=1793 loops=1)
                           Index Cond: (p_ci_pipelines_3.project_id = 278964)
                           Filter: (p_ci_pipelines_3.partition_id = ANY ('{114,115,116}'::bigint[]))
                           Buffers: shared hit=76 read=1740 dirtied=522
                           I/O Timings: read=694.487 write=0.000
                     ->  Index Scan using ci_pipelines_116_project_id_id_idx on gitlab_partitions_dynamic.ci_pipelines_116 p_ci_pipelines_4  (cost=0.43..1914.30 rows=1269 width=12) (actual time=0.384..250.164 rows=1267 loops=1)
                           Index Cond: (p_ci_pipelines_4.project_id = 278964)
                           Filter: (p_ci_pipelines_4.partition_id = ANY ('{114,115,116}'::bigint[]))
                           Buffers: shared hit=260 read=1122 dirtied=769
                           I/O Timings: read=212.717 write=0.000
Edited by Meir Benayoun

Merge request reports