Detect "stale" analyzer status and update related fields

What does this MR do and why?

Implements stale status tracking at the project and namespace level for the Security Inventory.

When a security analyzer absent in main branch pipelines, UpdateService now increments its consecutive_absence_count and transitions it to :stale after 3 consecutive absences. It also preserves the reference to the last build it ran so AppSec teams can investigate when the scan last occurred. Pipeline-excluded types and profile-covered types are exempt from this logic.

Stale counts are propagated up the group hierarchy: AncestorsUpdateService and RecalculateService now forward stale deltas so ancestor namespace statistics stay accurate when project pipelines change or projects are deleted/transferred. AdjustmentService also accounts for stale records in its drift-correction queries, preventing stale count divergence in group stats over time.

Feature flag

security_inventory_stale_analyzer_status is necessary until the upcoming FE changes are implemented. While it is off, it counts not_configured the old way. Essentially stale analyzers are counted and displayed in the UI as not configured ( see the gifs below ). When the FE is ready to show Stale status in the analyzer status breakdown bar component, then we will enable the FF, this will result in the accurate calculation of not_configured value.

References

Screenshots or screen recordings

Before After
"Not enabled" shown after first absence, link to the job lost

Once the second run completed, where DAST and CS were excluded via env variables. Still shows previous status and data about the last present scan.

statuses_once_2nd_run_completed.gif

After the 3rd run completed, with the same analyzers absent - UI shows "not enabled" but keeps the data about the last present job. The status is Stale in the GraphQL response

statuses_once_3rd_run_completed.gif

How to set up and validate locally

Given a project with Security Inventory showing enabled statuses. I used a simple Dockerized rack app that shows a hello world HTML page with auto dev ops enabled. Push a change, wait for it to complete the scans. Observe the scan statuses report actual Analyzer statuses ( success or failures )

Via skipping a scan on custom CI run

  1. Go to Build → Pipelines → Run pipeline.
  2. Under Variables, add the variables ( see below ).
  3. Click Run pipeline.
  4. Once finished, observe Sec Inventory shows the previous status, link to the last scan, and the time of the last scan
  5. Repeat steps 1 - 4
  6. Repeat steps 1 - 3. Once finished, observe:
    1. GraphQL responds with Stale status for the analyzers
    2. UI
      1. Observe the Sec Inventory shows not configured for the analyzers, it still shows link to the last scan, and the time of the last scan
  7. Trigger pipeline without disabling analyzers
  8. Once finished, observe the fresh statuses are shown ( success or failure ), with link to and data of the fresh scan
SAST_DISABLED=true
SECRET_DETECTION_DISABLED=true
DEPENDENCY_SCANNING_DISABLED=true
CONTAINER_SCANNING_DISABLED=true
DAST_DISABLED=true

Via DAST on demand scan

  1. Create On demand dast scan and run it.
  2. After DAST completes - observe the rest analyzers statuses display previous value, with link and time of the previous scan
  3. Repeat 2 times
  4. Observe the rest analyzers statuses display not configured, with link and time of the previous scan
  5. Trigger pipeline without disabling analyzers
  6. Once finished, observe the fresh statuses are shown ( success or failure ), with link to and data of the fresh scan

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Changelog: added
Edited by Vasyl Pedak

Merge request reports

Loading