Get rid of docker in docker requirement for Dependency Scanning
Customers have declined to use some of our scanning solutions because docker in docker is leveraged and that requires privileged runners, and it also prevents the runners from caching the images. The content of the docker-in-docker container is lost after the job run, and only
docker:stable is cached. As a result we are working to remove this requirement, and in this release we will remove the docker in docker requirement from the dependency scanning solution.
Problem to solve
Docker-in-docker is only required to be able to orchestrate the analyzers. One solution would be to port the code of the Dependency Scanning orchestrator directly into the runner.
NB: SAST and Dependency scanning share a common library and the same orchestration model, see SAST issue
See &971 (closed) for previous comments.
Update Dependency-Scanning.gitlab-ci.yml job template:
- add one job for each supported Dependency Scanning analyzer,
and leverage new CI variable
CI_PROJECT_REPOSITORY_LANGUAGESintroduced in !16477 (merged) to enable them when the project is compatible; share YAML code using extends
true by defaultfalse by default
- change the existing
dependency_scanningjob, enable it
Gemnasium would always run whereas other scanners would run selectively when the project is compatible.
This is similar to !16487 (diffs).
So assuming there's a shared definition named
gemnasium-dependency_scanning: extends: .analyzer image: name: "$DEPENDENCY_SCANNING_ANALYZER_IMAGE_PREFIX/gemnasium:$DS_MAJOR_VERSION" gemnasium-java-dependency_scanning: extends: .analyzer image: name: "$DEPENDENCY_SCANNING_ANALYZER_IMAGE_PREFIX/gemnasium-java:$DS_MAJOR_VERSION" only: variables: - '$CI_PROJECT_REPOSITORY_LANGUAGES =~ /java\b/' retire.js-dependency_scanning: extends: .analyzer image: name: "$DEPENDENCY_SCANNING_ANALYZER_IMAGE_PREFIX/retire.js:$DS_MAJOR_VERSION" only: variables: - '$CI_PROJECT_REPOSITORY_LANGUAGES =~ /javasript/' bundler-audit-dependency_scanning: extends: .analyzer image: name: "$DEPENDENCY_SCANNING_ANALYZER_IMAGE_PREFIX/bundler-audit:$DS_MAJOR_VERSION" only: variables: - '$CI_PROJECT_REPOSITORY_LANGUAGES =~ /ruby/'
DS_MAJOR_VERSION must be set to
- Update all related tests projects in https://gitlab.com/gitlab-org/security-products/tests
What does success look like, and how can we measure that?
There is no more orchestration layer for Dependency Scanning, instead each analyzer has its own job configured by the corresponding vendored template.