Change location fingerprint for Container Scanning

Problem to solve

Following the Design conclusion of Allow Container Scanning to scan multiple container images for a single change we need to update the way we store the location fingerprint for Container Scanning findings

Intended users

Internal: devopssecure & ~"devops::defend"

Further details

Proposal

Update the fingerprint used by container scanning so that the same vulnerability found in multiple distinct Docker images will not be de-duplicated and will instead be included in the final report, tied to the particular image it was found in. We'll also need to migrate all existing container scanning vulnerability database entries to use the new fingerprint. See this discussion for more details.

1. Update the fingerprint_data method in ee/lib/gitlab/ci/reports/security/locations/container_scanning.rb to use the following information:

   def fingerprint_data
     "#{docker_image_name_without_tag}:#{package_name}"
   end

   For example

   docker_image_name	docker_image_name_without_tag	package_name	fingerprint_data
   registry.gitlab.com/adamcohen/container-scanning-multi-image/alpine1-test-two-same-images:4d6bd9bcf4e64e15713e8edb1b921310a381e46f	registry.gitlab.com/adamcohen/container-scanning-multi-image/alpine1-test-two-same-images	musl	registry.gitlab.com/adamcohen/container-scanning-multi-image/alpine1-test-two-same-images:musl
   registry.gitlab.com/gitlab-org/security-products/dast/webgoat-8.0@sha256:bc09fe2e0721dfaeee79364115aeedf2174cce0947b9ae5fe7c33312ee019a4e	registry.gitlab.com/gitlab-org/security-products/dast/webgoat-8.0@sha256	nss	registry.gitlab.com/gitlab-org/security-products/dast/webgoat-8.0@sha256:nss
   alpine:3.7.3	alpine:3.7.3	musl	alpine:3.7.3:musl
   alpine:3.7	alpine:3.7	musl	alpine:3.7:musl

2. Introduce new method called new_fingerprint which will encapsulate new way of calculating fingerprint. update StoreReportService to use new_fingerprint for container scanning findings. Logic will be as following,

Rough implementation

    # rubocop: disable CodeReuse/ActiveRecord
    def create_or_find_vulnerability_finding(occurrence, create_params)
      return if occurrence.scanner.blank?

      find_params = {
        scanner: scanners_objects[occurrence.scanner.key],
        primary_identifier: identifiers_objects[occurrence.primary_identifier.key],
        location_fingerprint: occurrence.location.fingerprint
      }

      begin
        if occurrence.location.respond_to?(:fingerprint_v2)
          create_or_find_vulnerability_finding_v2(occurrence, create_params, find_params)
        else
          project
            .vulnerability_findings
            .create_with(create_params)
            .find_or_create_by!(find_params)
        end
      rescue ActiveRecord::RecordNotUnique
        project.vulnerability_findings.find_by!(find_params)
      rescue ActiveRecord::RecordInvalid => e
        Gitlab::ErrorTracking.track_and_raise_exception(e, create_params: create_params&.dig(:raw_metadata))
      end
    end


    def create_or_find_vulnerability_finding_v2(occurrence, create_params, find_params)
      project_with_vulns = project.vulnerability_findings

      vuln = project_with_vulns.find_by(find_params)

      if vuln.nil?
        find_params[:location_fingerprint] = occurrence.location.fingerprint_v2
        vuln = project_with_vulns.find_by(find_params)
        if vuln.nil?
         vuln = project_with_vulns.create!(create_params.merge(find_params))
        end
      else
        vuln.update_column(location_fingerprint: occurrence.location.fingerprint_v2)
      end

      vuln
    end

Other places fingerprint is used don't require update

• MR widget
• MergeReportsService

Permissions and Security

Documentation

Availability & Testing

Engineer to manually test to ensure multiple Docker images can be scanned with the same vulnerability and they show up as separate vulnerabillities

SET to look into adding an end to end test for container scanning, including multiple containers of the same type and vulnerability but check for distinct entries.

What does success look like, and how can we measure that?

Two Container Scanning findings with the same primary identifier (CVE) but coming from different docker images (image name must be different, not only the tags) are successfully kept in the database.

What is the type of buyer?

Is this a cross-stage feature?

@matt_wilson this is relevant for Category:Vulnerability Management of groupthreat insights.

Links / references