Skip to content

Populate missing dismissal information for vulnerabilities

What does this MR do?

This MR introduces a new background migration to populate missing dismissed_at and dismissed_by_id attributes for vulnerability records.

Related to #262112 (closed).

Database review

Currently, we have ~13k vulnerability records in Gitlab.com database which have to be updated by this migration.

EXPLAIN ANALYZE SELECT id FROM vulnerabilities WHERE state = 2 AND (dismissed_at IS NULL OR dismissed_by_id IS NULL)
 Seq Scan on public.vulnerabilities  (cost=0.00..443253.65 rows=141849 width=8) (actual time=1.485..6296.326 rows=13279 loops=1)
   Filter: (((vulnerabilities.dismissed_at IS NULL) OR (vulnerabilities.dismissed_by_id IS NULL)) AND (vulnerabilities.state = 2))
   Rows Removed by Filter: 4035176
   Buffers: shared read=98162 dirtied=9209 written=7603
   I/O Timings: read=4434.349 write=188.032

Since the batch size is 1k this means we will schedule 14 background jobs each of which will finish in 1 minute which makes it 14 minutes in total.

rake db:migrate
== 20201028160832 SchedulePopulateMissingDismissalInformationForVulnerabilities: migrating
== 20201028160832 SchedulePopulateMissingDismissalInformationForVulnerabilities: migrated (0.0642s)
== 20201102231721 AddTemporaryIndexToVulnerabilitiesTable: migrating ==========
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:vulnerabilities, :id, {:where=>"state = 2 AND (dismissed_at IS NULL OR dismissed_by_id IS NULL)", :name=>"temporary_index_vulnerabilities_on_id", :algorithm=>:concurrently})
   -> 0.0054s
-- add_index(:vulnerabilities, :id, {:where=>"state = 2 AND (dismissed_at IS NULL OR dismissed_by_id IS NULL)", :name=>"temporary_index_vulnerabilities_on_id", :algorithm=>:concurrently})
   -> 0.0132s
== 20201102231721 AddTemporaryIndexToVulnerabilitiesTable: migrated (0.0191s) =
rake db:migrate:down
== 20201028160832 SchedulePopulateMissingDismissalInformationForVulnerabilities: reverting
== 20201028160832 SchedulePopulateMissingDismissalInformationForVulnerabilities: reverted (0.0000s)
== 20201102231721 AddTemporaryIndexToVulnerabilitiesTable: reverting ==========
-- transaction_open?()
   -> 0.0000s
-- indexes(:vulnerabilities)
   -> 0.0056s
-- remove_index(:vulnerabilities, {:algorithm=>:concurrently, :name=>"temporary_index_vulnerabilities_on_id"})
   -> 0.0019s
== 20201102231721 AddTemporaryIndexToVulnerabilitiesTable: reverted (0.0079s) =

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • [-] Label as security and @ mention @gitlab-com/gl-security/appsec
  • [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • [-] Security reports checked/validated by a reviewer from the AppSec team
Edited by Mehmet Emin INAC

Merge request reports