Skip to content

Compare results from all pipelines related to change when evaluating Scan Result Policies

Why are we doing this work

It might be possible that the pipeline for target branch sha or source branch sha contains detached pipelines that do not include security jobs. Because of this we might incorrectly require approvals while they are not needed or not require approvals while they needed. With this change we want to change this behavior to get reports from all pipelines related to given source/target commit.

Release post

Scan result policies are a type of security policy that allows you to evaluate and block merge requests if particular rules are violated. Approvers may review and approve the change, or work with their development teams to address any issues (such as addressing critical security vulnerabilities).

To determine if there's a violation, we previously compared latest source and target branches to compare vulnerabilities and detect any new violations of policy rules. But, this may not capture vulnerabilities detected from scans running as a result of various pipeline sources. To increase accuracy, we will be comparing all of the latest completed pipelines of each pipeline source (with the exception of parent/child pipelines). This will ensure a more comprehensive evaluation and reduce the cases where approvals are required when it may be unexpected.

Use Cases to consider

  1. Project is using merge request pipelines and the results in these findings are not considered in the evaluation.
  2. A scheduled pipeline is being run overnight as it takes to long to run on every MR and slows developers down. The scheduled pipeline may include security scanners and may discover new findings that should be raised on existing MRs.
  3. A new CVE is posted related to a dependency (unrelated to the MR) that is already in the default branch. This results in a "new vulnerability" for that finding.
  4. A parent-child pipeline is used to separate scanners into an external/detached pipeline. (Out of scope for this issue, but we'll explore in future iteration. I think we can learn more about how prevalent this pattern is used with security scan jobs)

Relevant links

Non-functional requirements

  • Documentation:
  • Feature flag: Add feature flag to be able to enable/disable this behavior (default: false)
  • Performance:
  • Testing:

Implementation plan

  • backend Update Ci::Pipeline model to get latest pipelines for a given sha grouped by source
def self.latest_pipelines_per_source(sha)
  select('DISTINCT ON (source) *').where(sha: sha).order(:source, id: :desc)
end
  • backend Update Security::Finding model to get findings for given pipeline ids:
def for_pipelines(pipeline_ids)
  where(scan_id: Security::Scan.where(pipeline_id: pipeline_ids))
end
  • backend Update Security::ScanResultPolicies::SyncFindingsToApprovalRulesService to check if all latest pipelines for the latest pipeline sha are complete
  • backend Update Security::ScanResultPolicies::UpdateApprovalsService to consider the latest pipelines for target pipeline of MR and get the security findings
  • backend feature flag keep changes behind feature flag,

Verification steps

  • Create new project
  • Add .gitlab-ci.yml file that includes detached pipeline: 
      container_scanning:
    image: "busybox:latest"
    stage: test
    allow_failure: true
    artifacts:
      reports:
        container_scanning: gl-container-scanning-report.json
      paths: [gl-container-scanning-report.json]
    dependencies: []
    script:
      - wget -O gl-container-scanning-report.json https://gitlab.com/-/snippets/2438327/raw/main/gl-container-scanning-report-empty.json?inline=false
    rules:
      - if: $CONTAINER_SCANNING_DISABLED
        when: never
      - if: $CI_COMMIT_BRANCH

  other_job:
    image: busybox:latest
    stage: test
    script:
      - echo "This is other branch"

  detachedjob:
    rules:
      - when: always
    script:
      - echo "Hello from detached job"
  • Add Scan Result Policy (go to Security & Compliance -> Policies), click New Policy, select Scan Result Policies and add new policy and merge created MR: 
      name: Enforce check when there is at least 1 critical
  description: ''
  enabled: true
  actions:
  - type: require_approval
    approvals_required: 1
    group_approvers_ids:
    - # add your group ID
  rules:
  - type: scan_finding
    branches: []
    scanners: []
    vulnerabilities_allowed: 0
    severity_levels:
    - critical
    vulnerability_states:
    - newly_detected
  • Add new MR with change to .gitlab-ci.yml: 
      diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
  index 0cf690cc8622c2262dd1ea492e4019e40dc078d0..46167e7f57632551d085397fe6994a596d4c1ae3 100644
  --- a/.gitlab-ci.yml
  +++ b/.gitlab-ci.yml
  @@ -31,7 +31,7 @@ container_scanning:
      paths: [gl-container-scanning-report.json]
    dependencies: []
    script:
  -    - wget -O gl-container-scanning-report.json http://gdk.test:3000/-/snippets/22/raw/main/gl-container-scanning-report-empty.json?inline=false
  +    - wget -O gl-container-scanning-report.json http://gdk.test:3000/-/snippets/22/raw/main/gl-container-scanning-report-1-critical.json?inline=false
    rules:
      - if: $CONTAINER_SCANNING_DISABLED
        when: never
  • Verify if approvals in the MR are required
Edited by Grant Hickman