Get rid of docker in docker requirement for Dependency Scanning
Customers have declined to use some of our scanning solutions because docker in docker is leveraged and that requires privileged runners, and it also prevents the runners from caching the images. The content of the docker-in-docker container is lost after the job run, and only
docker:stable is cached. As a result we are working to remove this requirement, and in this release we will remove the docker in docker requirement from the dependency scanning solution.
Problem to solve
Docker-in-docker is only required to be able to orchestrate the analyzers. One solution would be to port the code of the Dependency Scanning orchestrator directly into the runner.
NB: SAST and Dependency scanning share a common library and the same orchestration model, see SAST issue
See &971 for previous comments.
Update Dependency-Scanning.gitlab-ci.yml job template:
- add one job for each supported Dependency Scanning analyzer,
and leverage new CI variable
CI_PROJECT_REPOSITORY_LANGUAGESintroduced in !16477 (merged) to enable them when the project is compatible; share YAML code using extends
DEPENDENCY_SCANNING_DISABLE_DINDvariable, true by default
- change the existing
dependency_scanningjob, enable it
Gemnasium would always run whereas other scanners would run selectively when the project is compatible.
This is similar to !16487 (diffs).
So assuming there's a shared definition named
gemnasium-dependency_scanning: extends: .analyzer image: name: "$DEPENDENCY_SCANNING_ANALYZER_IMAGE_PREFIX/gemnasium:2" gemnasium-java-dependency_scanning: extends: .analyzer image: name: "$DEPENDENCY_SCANNING_ANALYZER_IMAGE_PREFIX/gemnasium-java:2" only: variables: - '$CI_PROJECT_REPOSITORY_LANGUAGES =~ /java\b/' retire.js-dependency_scanning: extends: .analyzer image: name: "$DEPENDENCY_SCANNING_ANALYZER_IMAGE_PREFIX/retire.js:2" only: variables: - '$CI_PROJECT_REPOSITORY_LANGUAGES =~ /javasript/' bundler-audit-dependency_scanning: extends: .analyzer image: name: "$DEPENDENCY_SCANNING_ANALYZER_IMAGE_PREFIX/bundler-audit:2" only: variables: - '$CI_PROJECT_REPOSITORY_LANGUAGES =~ /ruby/'
We may also want to pin minor versions of the analyzers, and expose these versions as variables. See #9725
- Update all related tests projects in https://gitlab.com/gitlab-org/security-products/tests
What does success look like, and how can we measure that?
There is no more orchestration layer for Dependency Scanning, instead each analyzer has its own job configured by the corresponding vendored template.