Bulk create vulnerabilities when running vulnerability scans

What does this MR do and why?

This MR allows AdvisoryScanner class to pass in multiple findings from varying projects to the VulnerabilityScanning::CreateVulnerabilityService class when enabled. This change means that continuous vulnerability scans will become more efficient since we will have multiple vulnerabilities created per SQL transaction instead of a singular vulnerability per SQL transaction.

Some changes were needed to the error handling done in the creation service, so that errors from one project did not cause the entire batch of vulnerabilities to fail the creation process. The process for doing so goes as follows:

1. If an error is caught, the ID of the project affected is recorded and added to an array: project_ids_with_error.
2. If no findings could be created or if a SQL transaction error bubbles up to the execute method, the entire batch is discarded, and an error response is returned.
3. A response from the service can still be "successful" if at least one vulnerability was created. The caller should still inspect the project_ids_with_error array to analyze if any projects failed to get the vulnerability created.

Closes Make AdvisoryScanner ingest multiple finding maps (#429834 - closed) • Oscar Tovar • 16.7 • On track

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Bulk ingestion w/ CVS

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

1. Set up your GDK to sync with the external advisory database. Follow the instructions to enable license compliance, but run the modified script below instead for step 4.

./bin/rails runner 'PackageMetadata::AdvisoriesSyncWorker.perform_async'

2. Ensure that you've ingested the necessary advisory: advisory = PackageMetadata::Advisory.where(advisory_xid: "5912dc30-ecd8-49c9-b9c9-9923cae92053").first
3. Create a project with the attached gl-sbom-npm-npm.cdx.json and .gitlab-ci.yml files. Optionally, you can create more than one project with this structure to ensure that it can scale past one project.
4. Run the global advisory scan: Gitlab::VulnerabilityScanning::AdvisoryScanner.scan_projects_for(advisory, global: true)
5. Verify that the project vulnerabilities are no longer empty, and that a vulnerability with the advisory details has been created.

gl-sbom-npm-npm.cdx.json

{
  "bomFormat": "CycloneDX",
  "specVersion": "1.4",
  "serialNumber": "urn:uuid:09074d16-5dd6-483a-98cf-7956310d0609",
  "version": 1,
  "metadata": {
    "timestamp": "2023-09-12T06:52:39Z",
    "tools": [
      {
        "vendor": "GitLab",
        "name": "Gemnasium",
        "version": "4.4.0"
      }
    ],
    "authors": [
      {
        "name": "GitLab",
        "email": "support@gitlab.com"
      }
    ],
    "properties": [
      {
        "name": "gitlab:dependency_scanning:input_file",
        "value": "yarn.lock"
      },
      {
        "name": "gitlab:dependency_scanning:input_file:path",
        "value": "yarn.lock"
      },
      {
        "name": "gitlab:dependency_scanning:package_manager",
        "value": "yarn"
      },
      {
        "name": "gitlab:dependency_scanning:package_manager:name",
        "value": "yarn"
      },
      {
        "name": "gitlab:meta:schema_version",
        "value": "1"
      }
    ]
  },
  "components": [
    {
      "name": "directus",
      "version": "10.6.0",
      "purl": "pkg:npm/directus@10.6.0",
      "type": "library",
      "bom-ref": "pkg:npm/directus@10.6.0"
    },
    {
      "name": "directus",
      "version": "10.5.3",
      "purl": "pkg:npm/directus@10.5.3",
      "type": "library",
      "bom-ref": "pkg:npm/directus@10.5.3"
    },
    {
      "name": "string",
      "version": "3.3.3",
      "purl": "pkg:npm/string@3.3.3",
      "type": "library",
      "bom-ref": "pkg:npm/string@3.3.3"
    },
    {
      "name": "string-width",
      "version": "1.0.2",
      "purl": "pkg:npm/string-width@1.0.2",
      "type": "library",
      "bom-ref": "pkg:npm/string-width@1.0.2"
    },
    {
      "name": "string-width",
      "version": "2.1.1",
      "purl": "pkg:npm/string-width@2.1.1",
      "type": "library",
      "bom-ref": "pkg:npm/string-width@2.1.1"
    },
    {
      "name": "string-width",
      "version": "3.1.0",
      "purl": "pkg:npm/string-width@3.1.0",
      "type": "library",
      "bom-ref": "pkg:npm/string-width@3.1.0"
    },
    {
      "name": "bl",
      "version": "0.9.5",
      "purl": "pkg:npm/bl@0.9.5",
      "type": "library",
      "bom-ref": "pkg:npm/bl@0.9.5"
    },
    {
      "name": "bluebird",
      "version": "3.5.4",
      "purl": "pkg:npm/bluebird@3.5.4",
      "type": "library",
      "bom-ref": "pkg:npm/bluebird@3.5.4"
    },
    {
      "name": "bmp-js",
      "version": "0.0.3",
      "purl": "pkg:npm/bmp-js@0.0.3",
      "type": "library",
      "bom-ref": "pkg:npm/bmp-js@0.0.3"
    },
    {
      "name": "bn.js",
      "version": "4.11.8",
      "purl": "pkg:npm/bn.js@4.11.8",
      "type": "library",
      "bom-ref": "pkg:npm/bn.js@4.11.8"
    }
  ]
}

.gitlab-ci.yml

stages:
  - build

upload_cdx_reports:
  stage: build
  script:
    - echo "NOOP..."
  artifacts:
    untracked: false
    when: on_success
    expire_in: 30 days
    reports:
      cyclonedx: gl-sbom-*-*.cdx.json
    paths:
      - gl-sbom-*-*.cdx.json

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.