Get rid of docker in docker requirement for Dependency Scanning

Customers have declined to use some of our scanning solutions because docker in docker is leveraged and that requires privileged runners, and it also prevents the runners from caching the images. The content of the docker-in-docker container is lost after the job run, and only docker:stable is cached. As a result we are working to remove this requirement, and in this release we will remove the docker in docker requirement from the dependency scanning solution.

Problem to solve

Docker-in-docker is only required to be able to orchestrate the analyzers. One solution would be to port the code of the Dependency Scanning orchestrator directly into the runner.

NB: SAST and Dependency scanning share a common library and the same orchestration model, see SAST issue

See &971 (closed) for previous comments.

Intended users

• Persona: DevOps Engineer
• Persona: Software developer

Proposal

Update Dependency-Scanning.gitlab-ci.yml job template:

• add DS_ANALYZER_IMAGE_PREFIX variable
• add one job for each supported Dependency Scanning analyzer, and leverage new CI variable CI_PROJECT_REPOSITORY_LANGUAGES introduced in !16477 (merged) to enable them when the project is compatible; share YAML code using extends
• add DS_DISABLE_DIND variable, true by default false by default
• change the existing dependency_scanning job, enable it except when DS_DISABLE_DIND is true

Gemnasium would always run whereas other scanners would run selectively when the project is compatible.

This is similar to !16487 (diffs).

So assuming there's a shared definition named .dependency_scanning.

gemnasium-dependency_scanning:
  extends: .analyzer
  image:
    name: "$DEPENDENCY_SCANNING_ANALYZER_IMAGE_PREFIX/gemnasium:$DS_MAJOR_VERSION"

gemnasium-java-dependency_scanning:
  extends: .analyzer
  image:
    name: "$DEPENDENCY_SCANNING_ANALYZER_IMAGE_PREFIX/gemnasium-java:$DS_MAJOR_VERSION"
  only:
    variables:
      - '$CI_PROJECT_REPOSITORY_LANGUAGES =~ /java\b/'

retire.js-dependency_scanning:
  extends: .analyzer
  image:
    name: "$DEPENDENCY_SCANNING_ANALYZER_IMAGE_PREFIX/retire.js:$DS_MAJOR_VERSION"
  only:
    variables:
      - '$CI_PROJECT_REPOSITORY_LANGUAGES =~ /javasript/'

bundler-audit-dependency_scanning:
  extends: .analyzer
  image:
    name: "$DEPENDENCY_SCANNING_ANALYZER_IMAGE_PREFIX/bundler-audit:$DS_MAJOR_VERSION"
  only:
    variables:
      - '$CI_PROJECT_REPOSITORY_LANGUAGES =~ /ruby/'

Currently DS_MAJOR_VERSION must be set to 2.

See similar work for SAST

Documentation

• Update https://docs.gitlab.com/ee/user/application_security/dependency_scanning/index.html

Testing

• Update all related tests projects in https://gitlab.com/gitlab-org/security-products/tests

What does success look like, and how can we measure that?

There is no more orchestration layer for Dependency Scanning, instead each analyzer has its own job configured by the corresponding vendored template.

What is the type of buyer?

GitLab Ultimate

Links / references

Product Management - @NicoleSchwartz

• Release Post

This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.