Ingest agentic scan pipelines alongside CI pipelines on the default branch

What does this MR do and why?

With agentic_analyzer_security_ingestion on, default-branch findings can be lost today:

• The ingestion worker reads one Redis slot per project. Every finished pipeline overwrites it.
• When the CI pipeline and the scan pipeline both finish before the worker runs, only the later one is ingested.

This MR gives scan pipelines their own slot, and ingests both slots in one job.

Terms: a scan pipeline is a duo_workflow workload pipeline at the default-branch HEAD. The business logic security scan is the only producer today.

Part of the split of !246889. Tracker: https://gitlab.com/gitlab-org/gitlab/-/work_items/630266

Depends on / merge order

• Merge after !257196 (merged), which this MR targets.
• Sibling: !258793 (merged) (status and merged-results lookup). They touch different files and can merge in either order.

Before / after

Case	Tier	Before	After
CI and scan pipeline both finish before the worker runs	Ultimate	Only the later one is ingested	Both are ingested, in one job
Worker runs again for a later scan pipeline	Ultimate	n/a	The CI pipeline is not ingested a second time

Flag off / on

• Flag off: unchanged. The scan slot is never written or read. The regular slot is never freed.
• Flag on: scan pipelines use a new slot, latest_agentic_scan_pipeline, with a 1-day expiry.
• Flag on: the worker ingests the regular slot first, then the scan slot.
• Flag on: both slots are freed with a compare-and-delete. A newer pipeline stored meanwhile stays for the next run.

Rollback: turn the flag off. A scan already waiting in its slot is not ingested, and expires within a day.

How to review

One commit. Start with store_security_reports_by_project_worker.rb, then store_scans_service.rb.

Trade-offs

• A scan pipeline that keeps failing to ingest keeps its slot. Later jobs then raise, for up to a day.
• The regular pipeline is ingested first, so its findings still land.
• A separate job per pipeline was rejected. It would let two ingestions for one project run at once.
• default_branch_scan? accepts any duo_workflow workload pipeline at the default-branch HEAD. It can't tell the business logic scan from another Duo flow.

Reviewers

This changes the vulnerability ingestion worker. It needs a vulnerability management reviewer.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

🤖 Generated with Claude Code