Skip to content

Load artifacts based on the finding metadata

What does this MR do?

This MR introduces a new way to load report artifacts to build data for the "pipeline security tab". Before this MR, we were trying to load all the artifacts into memory which was causing timeout errors for the pipelines with lots of security scans. With this MR, we are trying to load only the necessary artifacts into memory for the given parameters(filters and pagination) therefore this will fix the timeout issues.

Here is a diagram to describe the overall idea;

Untitled_Diagram

Related to #267021 (closed).

Database migrations

This MR introduces a new index for vulnerability_occurrences table on project_fingerprint column.

rake db:migrate:up

== 20201110035029 CreatedIndexForVulnerabilityOccurrencesOnProjectFingerprint: migrating
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:vulnerability_occurrences, :project_fingerprint, {:name=>"index_vulnerability_occurrences_on_project_fingerprint", :algorithm=>:concurrently})
   -> 0.0043s
-- add_index(:vulnerability_occurrences, :project_fingerprint, {:name=>"index_vulnerability_occurrences_on_project_fingerprint", :algorithm=>:concurrently})
   -> 0.0046s
== 20201110035029 CreatedIndexForVulnerabilityOccurrencesOnProjectFingerprint: migrated (0.0094s)

rake db:migrate:down

== 20201110035029 CreatedIndexForVulnerabilityOccurrencesOnProjectFingerprint: reverting
-- transaction_open?()
   -> 0.0000s
-- indexes(:vulnerability_occurrences)
   -> 0.0039s
-- remove_index(:vulnerability_occurrences, {:algorithm=>:concurrently, :name=>"index_vulnerability_occurrences_on_project_fingerprint"})
   -> 0.0022s
== 20201110035029 CreatedIndexForVulnerabilityOccurrencesOnProjectFingerprint: reverted (0.0065s)

Screenshots

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • [-] Label as security and @ mention @gitlab-com/gl-security/appsec
  • [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • [-] Security reports checked/validated by a reviewer from the AppSec team
Edited by Mehmet Emin INAC

Merge request reports