Skip to content

BG Migration to attach findings to their vulnerability links

What does this MR do and why?

In the process of enabling the :deprecate_vulnerability_feedback FF. This MR ensures that currently existing vulnerability feedback records have a Vulnerability::IssueLink or Vulnerability::MergeRequestLink between their associated Finding and Issue/MergeRequest records so that we can relate between them without the Vulnerability::Feedback model.

This MR cycles through all the existing Vulnerability::Feedback records of issue and merge request type and configures the IssueLink and MergeRequestLink records. In the process of doing this, it is necessary to create Vulnerability records as part of the deprecation is a change of approach in that findings that are interacted with have this interaction related to a Vulnerability record that is listed as present_on_default_branch: false. (Vulnerability::Feedback models currently relate directly to the Finding)

Note: During implementation I noticed the possibility of Vulnerability::Feedback issue records that relate to a null issue. The code actively filters off these records as there's no real resolution.

How to set up and validate locally

  1. With :deprecate_vulnerability_feedback disabled, use the pipeline security tab to create issues from findings in a pipeline
  2. Execute the background migration
  3. Note the new IssueLink records between those issues and their respective, newly create Vulnerability issues.
  4. With :deprecate_vulnerability_feedback disabled, use the pipeline security tab to create merge requests from findings in a pipeline (this can be a bit hard to reproduce as very few findings provide a remediation to provide this functionality)
  5. Execute the background migration
  6. Note the new MergeRequestLink records between those merge requests and their respective, newly create Vulnerability issues.

Query Timings

Primary Iteration Query Example

2.835 seconds https://postgres.ai/console/gitlab/gitlab-production-tunnel-pg12/sessions/14772/commands/51655

SELECT "vulnerability_feedback".*
FROM   "vulnerability_feedback"
WHERE  "vulnerability_feedback"."id" IN ( 100000, 100001, 100002, 100003,
                                          100004, 100005, 100006, 100007,
                                          100008, 100009, 100010, 100011,
                                          100012, 100013, 100014, 100015,
                                          100016, 100017, 100018, 100019,
                                          100020, 100021, 100022, 100023,
                                          100024, 100025, 100026, 100027,
                                          100028, 100029, 100030, 100031,
                                          100032, 100033, 100034, 100035,
                                          100036, 100037, 100038, 100039,
                                          100040, 100041, 100042, 100043,
                                          100044, 100045, 100046, 100047,
                                          100048, 100049, 100050, 100051,
                                          100052, 100053, 100054, 100055,
                                          100056, 100057, 100058, 100059,
                                          100060, 100061, 100062, 100063,
                                          100064, 100065, 100066, 100067,
                                          100068, 100069, 100070, 100071,
                                          100072, 100073, 100074, 100075,
                                          100076, 100077, 100078, 100079,
                                          100080, 100081, 100082, 100083,
                                          100084, 100085, 100086, 100087,
                                          100088, 100089, 100090, 100091,
                                          100092, 100093, 100094, 100095,
                                          100096, 100097, 100098, 100099,
                                          100100, 100101, 100102, 100103,
                                          100104, 100105, 100106, 100107,
                                          100108, 100109, 100110, 100111,
                                          100112, 100113, 100114, 100115,
                                          100116, 100117, 100118, 100119,
                                          100120, 100121, 100122, 100123,
                                          100124, 100125, 100126, 100127,
                                          100128, 100129, 100130, 100131,
                                          100132, 100133, 100134, 100135,
                                          100136, 100137, 100138, 100139,
                                          100140, 100141, 100142, 100143,
                                          100144, 100145, 100146, 100147,
                                          100148, 100149, 100150, 100151,
                                          100152, 100153, 100154, 100155,
                                          100156, 100157, 100158, 100159,
                                          100160, 100161, 100162, 100163,
                                          100164, 100165, 100166, 100167,
                                          100168, 100169, 100170, 100171,
                                          100172, 100173, 100174, 100175,
                                          100176, 100177, 100178, 100179,
                                          100180, 100181, 100182, 100183,
                                          100184, 100185, 100186, 100187,
                                          100188, 100189, 100190, 100191,
                                          100192, 100193, 100194, 100195,
                                          100196, 100197, 100198, 100199, 100200
                                        )
       AND ( EXISTS (SELECT 1
                     FROM   vulnerability_occurrences
                     WHERE  vulnerability_occurrences.uuid =
                            vulnerability_feedback.finding_uuid :: VARCHAR)
              OR EXISTS (SELECT 1
                         FROM   vulnerability_occurrences
                         WHERE  vulnerability_occurrences.project_fingerprint =
vulnerability_feedback.project_fingerprint :: bytea)
OR EXISTS (SELECT 1
FROM   security_findings
WHERE  security_findings.uuid =
vulnerability_feedback.finding_uuid) ) 

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #384222 (closed)

Edited by Gregory Havenga

Merge request reports

Loading