Get rid of docker in docker requirement for Dependency Scanning
Customers have declined to use some of our scanning solutions because docker in docker is leveraged and that requires privileged runners, and it also prevents the runners from caching the images. The content of the docker-in-docker container is lost after the job run, and only `docker:stable` is cached. As a result we are working to remove this requirement, and in this release we will remove the docker in docker requirement from the dependency scanning solution.
### Problem to solve
Docker-in-docker is only required to be able to orchestrate the analyzers. One solution would be to port the code of the Dependency Scanning orchestrator directly into the runner.
NB: SAST and Dependency scanning share a common library and the same orchestration model, [see SAST issue](https://gitlab.com/gitlab-org/gitlab-ee/issues/10796)
See https://gitlab.com/groups/gitlab-org/-/epics/971 for previous comments.
### Intended users
- ~"Persona: DevOps Engineer"
- ~"Persona: Software developer"
### Proposal
Update [Dependency-Scanning.gitlab-ci.yml](https://gitlab.com/gitlab-org/gitlab-foss/blob/master/lib/gitlab/ci/templates/Security/Dependency-Scanning.gitlab-ci.yml) job template:
- add `DS_ANALYZER_IMAGE_PREFIX` variable
- add one job for each supported Dependency Scanning analyzer,
and leverage new CI variable `CI_PROJECT_REPOSITORY_LANGUAGES` introduced in https://gitlab.com/gitlab-org/gitlab/merge_requests/16477 to enable them when the project is compatible; share YAML code using extends
- add `DS_DISABLE_DIND` variable, ~~true by default~~ false by default
- change the existing `dependency_scanning` job, enable it `except` when `DS_DISABLE_DIND` is true
Gemnasium would always run whereas other scanners would run selectively when the project is compatible.
This is similar to https://gitlab.com/gitlab-org/gitlab/merge_requests/16487/diffs.
So assuming there's a shared definition named `.dependency_scanning`.
```yaml
gemnasium-dependency_scanning:
extends: .analyzer
image:
name: "$DEPENDENCY_SCANNING_ANALYZER_IMAGE_PREFIX/gemnasium:$DS_MAJOR_VERSION"
gemnasium-java-dependency_scanning:
extends: .analyzer
image:
name: "$DEPENDENCY_SCANNING_ANALYZER_IMAGE_PREFIX/gemnasium-java:$DS_MAJOR_VERSION"
only:
variables:
- '$CI_PROJECT_REPOSITORY_LANGUAGES =~ /java\b/'
retire.js-dependency_scanning:
extends: .analyzer
image:
name: "$DEPENDENCY_SCANNING_ANALYZER_IMAGE_PREFIX/retire.js:$DS_MAJOR_VERSION"
only:
variables:
- '$CI_PROJECT_REPOSITORY_LANGUAGES =~ /javasript/'
bundler-audit-dependency_scanning:
extends: .analyzer
image:
name: "$DEPENDENCY_SCANNING_ANALYZER_IMAGE_PREFIX/bundler-audit:$DS_MAJOR_VERSION"
only:
variables:
- '$CI_PROJECT_REPOSITORY_LANGUAGES =~ /ruby/'
```
Currently `DS_MAJOR_VERSION` must be set to `2`.
See [similar work for SAST](https://gitlab.com/gitlab-org/gitlab-ee/issues/10796)
### Documentation
- [x] Update https://docs.gitlab.com/ee/user/application_security/dependency_scanning/index.html
### Testing
- [ ] Update all related tests projects in https://gitlab.com/gitlab-org/security-products/tests
### What does success look like, and how can we measure that?
There is no more orchestration layer for Dependency Scanning, instead each analyzer has its own job configured by the corresponding vendored template.
### What is the type of buyer?
~"GitLab Ultimate"
### Links / references
### Product Management - @NicoleSchwartz
* [x] [Release Post](https://gitlab.com/gitlab-com/www-gitlab-com/merge_requests/32379)
<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->
*This page may contain information related to upcoming products, features and functionality.
It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes.
Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.*
<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->
issue