Skip to content

Ignore tag pipeline for scan result policy comparison

Sashi Kumar Kumaresan requested to merge sk/421048 into master

What does this MR do and why?

Addresses #421048 (closed)

This MR fixes a behaviour in selection pipelines for scan result policies in which if a project has repository tags with pipeline and if they don't have security scan jobs, the tag pipeline would be considered for comparison resulting in incorrect approval enforced in MR.

Database Query

SELECT
    max(id) as id 
FROM
    "ci_pipelines" 
WHERE
    "ci_pipelines"."project_id" = 278964 
    AND (
        "ci_pipelines"."status" IN (
            'success','failed','canceled','skipped'
        )
    ) 
    AND "ci_pipelines"."ref" = 'master'
    AND "ci_pipelines"."tag" = false
    AND "ci_pipelines"."sha" = '6418b07f2e90539ce87dbe51b8624d2e486b6ee0' 
    AND (
        "ci_pipelines"."source" IN (
            1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 15
        ) 
        OR "ci_pipelines"."source" IS NULL
    ) 
GROUP BY
    "ci_pipelines"."source"

EXPLAIN Output

 Aggregate  (cost=3.75..3.77 rows=1 width=8) (actual time=45.993..45.998 rows=2 loops=1)
   Group Key: ci_pipelines.source
   Buffers: shared hit=3 read=21 dirtied=1
   I/O Timings: read=45.570 write=0.000
   ->  Sort  (cost=3.75..3.76 rows=1 width=8) (actual time=45.976..45.978 rows=3 loops=1)
         Sort Key: ci_pipelines.source
         Sort Method: quicksort  Memory: 25kB
         Buffers: shared hit=3 read=21 dirtied=1
         I/O Timings: read=45.570 write=0.000
         ->  Index Scan using index_ci_pipelines_on_project_id_and_sha on public.ci_pipelines  (cost=0.70..3.74 rows=1 width=8) (actual time=14.905..45.937 rows=3 loops=1)
               Index Cond: ((ci_pipelines.project_id = 278964) AND ((ci_pipelines.sha)::text = '6418b07f2e90539ce87dbe51b8624d2e486b6ee0'::text))
               Filter: ((NOT ci_pipelines.tag) AND ((ci_pipelines.ref)::text = 'master'::text) AND ((ci_pipelines.status)::text = ANY ('{success,failed,canceled,skipped}'::text[])) AND ((ci_pipelines.source = ANY ('{1,2,3,4,5,6,7,8,10,11,15}'::integer[])) OR (ci_pipelines.source IS NULL)))
               Rows Removed by Filter: 14
               Buffers: shared read=21 dirtied=1
               I/O Timings: read=45.570 write=0.000

Time: 55.006 ms
  - planning: 8.921 ms
  - execution: 46.085 ms
    - I/O read: 45.570 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 3 (~24.00 KiB) from the buffer pool
  - reads: 21 (~168.00 KiB) from the OS file cache, including disk I/O
  - dirtied: 1 (~8.00 KiB)
  - writes: 0

Old Plan

 Aggregate  (cost=3.75..3.77 rows=1 width=8) (actual time=78.052..78.057 rows=2 loops=1)
   Group Key: ci_pipelines.source
   Buffers: shared hit=3 read=21 dirtied=2
   I/O Timings: read=77.587 write=0.000
   ->  Sort  (cost=3.75..3.75 rows=1 width=8) (actual time=78.039..78.041 rows=5 loops=1)
         Sort Key: ci_pipelines.source
         Sort Method: quicksort  Memory: 25kB
         Buffers: shared hit=3 read=21 dirtied=2
         I/O Timings: read=77.587 write=0.000
         ->  Index Scan using index_ci_pipelines_on_project_id_and_sha on public.ci_pipelines  (cost=0.70..3.74 rows=1 width=8) (actual time=27.555..77.993 rows=5 loops=1)
               Index Cond: ((ci_pipelines.project_id = 278964) AND ((ci_pipelines.sha)::text = '6418b07f2e90539ce87dbe51b8624d2e486b6ee0'::text))
               Filter: (((ci_pipelines.status)::text = ANY ('{success,failed,canceled,skipped}'::text[])) AND ((ci_pipelines.source = ANY ('{1,2,3,4,5,6,7,8,10,11,15}'::integer[])) OR (ci_pipelines.source IS NULL)))
               Rows Removed by Filter: 12
               Buffers: shared read=21 dirtied=2
               I/O Timings: read=77.587 write=0.000

Time: 85.323 ms
  - planning: 7.194 ms
  - execution: 78.129 ms
    - I/O read: 77.587 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 3 (~24.00 KiB) from the buffer pool
  - reads: 21 (~168.00 KiB) from the OS file cache, including disk I/O
  - dirtied: 2 (~16.00 KiB)
  - writes: 0

How to set up and validate locally

  • Create a project with security scan jobs that introduces vulnerabilities and make it available except for tags type by adding:
  except:
    - tags
  • Create a scan result policy to enforce approval on newly detected vulnerabilities
  • Create a tag and make sure that the security scan does not run for the tag pipeline.
  • After the tag pipeline is complete, create a MR that does not introduce any new vulnerability (update README)
  • Observe that the MR does not require approval

Example Project: https://gitlab.com/gitlab-org/govern/security-policies/sashis-test-group/test-419789

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Sashi Kumar Kumaresan

Merge request reports