Dependency Scanning QA fails b/c DS_DEFAULT_ANALYZERS has changed
Summary
When updating the job template for Dependency Scanning
in order to get rid of Docker-in-Docker,
CI variable DS_DEFAULT_ANALYZERS
has been set to a value
that doesn't match the defaults set in depenency-scanning,
resulting in a change in the generated reports.
This bug affects developers working on Dependency Scanning but should have NO significant impact on users.
This have no consequence when Docker-in-Docker is disabled, which will eventually become the default.
Further details
Old value in dependency-scanning CI config:
DS_DEFAULT_ANALYZERS: bundler-audit,retire.js,gemnasium,gemnasium-maven,gemnasium-python
New value in dependency scanning job template:
DS_DEFAULT_ANALYZERS: "gemnasium, retire.js, gemnasium-python, gemnasium-maven, bundler-audit"`
Also, this doesn't match the QA jobs of the Gemnasium analyzer, making all pipelines fail for this project, when testing the gemnasium
Docker image against test project ruby-bundler. Actually this how this change was surfaced.
See failing QA job: https://gitlab.com/gitlab-org/security-products/tests/ruby-bundler/-/jobs/354304580
With Docker-in-Docker
If DS_DISABLE_DIND
is false
(current default value), there's one CI job that runs the dependency-scanning
project, which in turn runs the analyzers, using Docker-in-Docker.
Dependency Scanning is built on top of the common library. When scanning a project, it iterates over the analyzers (docker images) and merge their reports, respecting the order of CLI flag DS_DEFAULT_ANALYZERS
. The function used to merge the reports appends all the vulnerabilities, sort them by severity and compare key, and then remove the dupes sharing the same location and some identifier. So if there are duplicates between the vulnerabilities found by bundler-audit and the ones found by gemnasium, the result depends on which analyzer runs first. If bundler-audit is first, its occurrences "win" over gemnasium's when looking for duplicates.
Without Docker-in-Docker
If DS_DISABLE_DIND
is forced to true
, there are multiple CI jobs corresponding to the Dependency Scanning analyzers compatible with the project, based on CI_PROJECT_REPOSITORY_LANGUAGES
.
In this case DS_DEFAULT_ANALYZERS
has no effect.
TODO: link to implementation, in the Rails backend.
Steps to reproduce
In test project ruby-bundler, trigger a pipeline for branch caneldem-master-patch-80308
with these settings:
SAST_DISABLED: true
CONTAINER_SCANNING_DISABLED: true
DAST_DISABLED: true
LICENSE_MANAGEMENT_DISABLED: true
DS_DEFAULT_ANALYZERS:
DS_ANALYZER_IMAGES: registry.gitlab.com/gitlab-org/security-products/analyzers/bundler-audit:2,registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium:2
See https://gitlab.com/gitlab-org/security-products/tests/ruby-bundler/-/jobs/355170843
What is the current bug behavior?
gemnasium
takes precedence over bundler-audit
.
What is the expected correct behavior?
bundler-audit
takes precedence over gemnasium
.
Relevant logs and/or screenshots
Again, see https://gitlab.com/gitlab-org/security-products/tests/ruby-bundler/-/jobs/355170843
Possible fixes
2 options:
- Accept this change, keep the job template as it is, and update CI config of DS analyzers gemnasium and bundler-audit to align with that change. This is quite cheap.
- Update job template and restore the initial value of
DS_DEFAULT_ANALYZERS
, and revert the change of the expected report in test project ruby-bundler. BUT this change wouldn't be applied right away because the job template belongs to the gitlab project, so in the meantime a workaround would be to forceDS_DEFAULT_ANALYZERS
in the test project.
The latter is more expensive.