Change security inventory data source to security scans (!238568) · Merge requests · GitLab.org / GitLab

What does this MR do and why?

Uses a scan-based data source for the security inventory. Security::AnalyzersStatus::UpdateService updates analyzer statuses based on security_scans instead of CI job artifacts.

This change is required to show SARIF report results in the inventory. Additionally, this allows us to set failed status when the scan has report_error / preparation_failed.

Why a separate `IngestionSubscriberWorker`?

The existing PipelineAnalyzersStatusUpdateWorker has a perform(pipeline_id) signature used by the still active Pipeline.after_transition hook. Changing that would break in-flight Sidekiq jobs queued under the old shape. The new IngestionSubscriberWorker subscribes to the event and forwards to the existing worker via perform_async(pipeline.id), keeping backward compatibility. This can be removed when the FF is globally enabled.

How to set up and validate locally

Make sure the sarif_ingestion FF is on:
```
Feature.disable(:sarif_ingestion)
```
Make sure the analyzers_status_from_security_scans FF is off:
```
Feature.disable(:analyzers_status_from_security_scans)
```

In a project .gitlab-ci.yml add a job with SARIF artifact:

job1:
  stage: test
  script:
    - echo hello
  artifacts:
    access: 'developer'
    reports:
      sarif: sarif.json

Add the following content to the project root level sarif.json file:

sarif.json

{
  "version": "2.1.0",
  "$schema": "https://raw.githubusercontent.com/oasis-tcs/sarif-spec/master/Schemata/sarif-schema-2.1.0.json",
  "runs": [
    {
      "tool": {
        "driver": {
          "name": "Trivy",
          "version": "0.50.0",
          "informationUri": "https://github.com/aquasecurity/trivy",
          "rules": [
            {
              "id": "CVE-2021-44228",
              "name": "Log4Shell",
              "shortDescription": {
                "text": "Apache Log4j2 remote code execution"
              },
              "fullDescription": {
                "text": "Apache Log4j2 2.0-beta9 through 2.15.0 JNDI features used in configuration, log messages, and parameters do not protect against attacker controlled LDAP."
              },
              "helpUri": "https://nvd.nist.gov/vuln/detail/CVE-2021-44228",
              "properties": {
                "security-severity": "10.0"
              }
            },
            {
              "id": "CVE-2022-22965",
              "name": "Spring4Shell",
              "shortDescription": {
                "text": "Spring Framework RCE via Data Binding"
              },
              "properties": {
                "security-severity": "9.8"
              }
            }
          ]
        }
      },
      "originalUriBaseIds": {
        "ROOTPATH": {
          "uri": "file:///app/"
        }
      },
      "results": [
        {
          "ruleId": "CVE-2021-44228",
          "ruleIndex": 0,
          "level": "error",
          "message": {
            "text": "log4j-core 2.14.1 is vulnerable to CVE-2021-44228 (Log4Shell)."
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "pom.xml",
                  "uriBaseId": "ROOTPATH"
                },
                "region": {
                  "startLine": 42,
                  "endLine": 42,
                  "startColumn": 1,
                  "endColumn": 80
                }
              }
            }
          ]
        },
        {
          "ruleId": "CVE-2022-22965",
          "ruleIndex": 1,
          "level": "error",
          "message": {
            "text": "spring-beans 5.3.15 is vulnerable to CVE-2022-22965 (Spring4Shell)."
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "build.gradle",
                  "uriBaseId": "ROOTPATH"
                },
                "region": {
                  "startLine": 17,
                  "endLine": 17
                }
              }
            }
          ]
        }
      ]
    },
    {
      "tool": {
        "driver": {
          "name": "Semgrep",
          "version": "1.45.0",
          "informationUri": "https://semgrep.dev",
          "rules": [
            {
              "id": "ruby.lang.security.sqli.tainted-sql-string",
              "name": "TaintedSqlString",
              "shortDescription": {
                "text": "Possible SQL injection via tainted string interpolation"
              },
              "properties": {
                "tags": ["security", "CWE-89", "CWE-89: Improper Neutralization of Special Elements used in an SQL Command"]
              }
            },
            {
              "id": "generic.secrets.hardcoded-password",
              "name": "HardcodedPassword",
              "shortDescription": {
                "text": "Hardcoded credential found"
              },
              "properties": {
                "tags": ["security", "CWE-798", "CWE-798: Use of Hard-coded Credentials"]
              }
            }
          ]
        }
      },
      "originalUriBaseIds": {
        "ROOTPATH": {
          "uri": "file:///app/"
        }
      },
      "results": [
        {
          "ruleId": "ruby.lang.security.sqli.tainted-sql-string",
          "ruleIndex": 0,
          "level": "warning",
          "message": {
            "text": "Untrusted input flows into a SQL string built by interpolation."
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "app/finders/user_finder.rb",
                  "uriBaseId": "ROOTPATH"
                },
                "region": {
                  "startLine": 22,
                  "endLine": 22,
                  "startColumn": 5,
                  "endColumn": 60
                }
              }
            }
          ]
        },
        {
          "ruleId": "generic.secrets.hardcoded-password",
          "ruleIndex": 1,
          "level": "error",
          "message": {
            "text": "Hardcoded credential detected in source."
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "config/initializers/secrets.rb",
                  "uriBaseId": "ROOTPATH"
                },
                "region": {
                  "startLine": 8,
                  "endLine": 8,
                  "startColumn": 1,
                  "endColumn": 40
                }
              }
            }
          ]
        }
      ]
    },
    {
      "tool": {
        "driver": {
          "name": "Flawfinder",
          "version": "2.0.19",
          "informationUri": "https://dwheeler.com/flawfinder/",
          "rules": [
            {
              "id": "FF1001",
              "name": "BufferOverflow",
              "shortDescription": {
                "text": "Statically-sized arrays can be improperly restricted, leading to buffer overflows."
              },
              "relationships": [
                {
                  "target": {
                    "id": "CWE-119",
                    "toolComponent": {
                      "name": "CWE"
                    }
                  },
                  "kinds": ["superset"]
                }
              ]
            }
          ],
          "supportedTaxonomies": [
            {
              "name": "CWE",
              "index": 0
            }
          ]
        }
      },
      "originalUriBaseIds": {
        "ROOTPATH": {
          "uri": "file:///src/"
        }
      },
      "results": [
        {
          "ruleId": "FF1001",
          "ruleIndex": 0,
          "level": "warning",
          "message": {
            "text": "strcpy used; potentially unsafe buffer write."
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "src/parser.c",
                  "uriBaseId": "ROOTPATH"
                },
                "region": {
                  "startLine": 145,
                  "endLine": 145,
                  "startColumn": 3,
                  "endColumn": 35
                }
              }
            }
          ]
        }
      ]
    },
  ]
}

Run a pipeline and wait for it to finish, then check Security::AnalyzerProjectStatus for the project:
```
Security::AnalyzerProjectStatus.where(project: p).pluck(:analyzer_type, :status)
```

Enable the analyzers_status_from_security_scans FF:

Feature.enable(:analyzers_status_from_security_scans)

Re-run the SARIF pipeline. Once the pipeline completes re-check:

Security::AnalyzerProjectStatus.where(project: <project>).pluck(:analyzer_type, :status)
# => [["sast", "success"], ...] 
# Expect sast, dependency_scanning, secret_detection_pipeline_based...

Query plans

pipeline_scans:

Raw SQL

SELECT
    "security_scans".*
FROM
    "security_scans"
WHERE
    "security_scans"."pipeline_id" = 2553032240
    AND "security_scans"."latest" = TRUE
    AND "security_scans"."status" NOT IN (6, 0, 4)
ORDER BY
    "security_scans"."created_at" ASC,
    "security_scans"."id" ASC

Query plan

See details here.

 Sort  (cost=10.14..10.14 rows=1 width=102) (actual time=9.237..9.238 rows=6 loops=1)
   Sort Key: security_scans.created_at, security_scans.id
   Sort Method: quicksort  Memory: 25kB
   Buffers: shared hit=6 read=8 dirtied=1
   WAL: records=2 fpi=1 bytes=8093
   ->  Index Scan using index_security_scans_on_length_of_warnings on public.security_scans  (cost=0.57..10.13 rows=1 width=102) (actual time=7.167..9.200 rows=6 loops=1)
         Index Cond: (security_scans.pipeline_id = '2553032240'::bigint)
         Filter: (security_scans.latest AND (security_scans.status <> ALL ('{6,0,4}'::integer[])))
         Rows Removed by Filter: 0
         Buffers: shared read=8 dirtied=1
         WAL: records=2 fpi=1 bytes=8093
Settings: effective_cache_size = '338688MB', jit = 'off', random_page_cost = '1.5', work_mem = '100MB', seq_page_cost = '4'

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #599288 (closed)

Edited Jun 04, 2026 by Gal Katz

Change security inventory data source to security scans

What does this MR do and why?

Why a separate IngestionSubscriberWorker?

How to set up and validate locally

Query plans

MR acceptance checklist

Merge request reports

Why a separate `IngestionSubscriberWorker`?