Remove rules:exists from Dependency Scanning jobs
Release notes
Problem to solve
Dependency Scanning jobs are skipped when the contents of the git repo being scanned doesn't match their rules:exists parameter but are downsides to this optimization:
- It makes it impossible to notify users about what's not supported at the moment, and link to user docs and/or the relevant featureaddition issue. (This could be done in the Rails backend but would create maintenance issues.)
- UX is inconsistent because sometimes scanning jobs are triggered even though there's nothing to scan.
- The files that match
rules:exists
are excluded by theDS_EXCLUDED_PATHS
variables. - GitLab matches a maximum of 10,000 exists patterns or file paths, and the job is triggered regardless of what's in the repo.
- The files that match
- It makes it harder to report a bug about a project not being scanned, because the scanning job is simply missing. It would be easier if users or support engineers could link to the job where the scan isn't happening.
- It creates maintenance issues because the Dependency Scanning CI template and and the corresponding QA template must be updated when the analyzer supports new file, resulting in a change that spreads over 3 repositories, and that is really hard to coordinate.
- It creates a gap between job integration tests and image integration tests because the latter don't cover
rules:exists
. It forces developers to maintain expensive job integration tests.
Running scanning jobs that aren't compatible with a GitLab project is potentially a waste of bandwidth when the Docker image of the analyzer is quite big and there's no cache. That said, users can still optimize their pipelines and skip analyzers using DS_EXCLUDED_ANALYZERS
.
Proposal
Remove rules:exists parameters from the Dependency Scanning jobs:
- in the CI template users include
- in the QA template developers use for job integration tests
Possible follow-up issues:
- Write an info or warning message to the log, to suggest users they add the scanner name to
DS_EXCLUDED_ANALYZERS
. - Update
DS_EXCLUDED_ANALYZERS
automatically in order to skip the job. The problem is, if users use another language or package manager in their project, then it won't be scanned automatically.
Counter-proposal: change the rules:exists
param to make it more generic, so that it doesn't change much, and so that it catches unsupported package managers. For instance, change the rules:exists
of gemnasium-maven
to match Java and Scala files, instead of POM XML, Sbt build files, and Gradle build files.
Intended users
TBD
User experience goal
As a user having a Java or Python project, when my project is not scanned by GitLab Dependency Scanning, I go to the log of the gemnasium-maven
or gemnasium-python
job to learn why it's not supported.
Permissions and Security
No change
Documentation
The new behavior must be documented in Dependency Scanning documentation.
Availability & Testing
- New tests ensures that scanning jobs are triggered even when the files aren't supported
Also, it becomes possible to refactor some of the existing tests:
- Some of the job integration tests can be converted to image integration tests; see #330093 (comment 585331906)
- Existing job integrations tests can be implemented using child pipelines, and external test projects can be replaced with fixtures that we keep in the git repo of the analyzer project; see #297361 (comment 607755557)
What does success look like, and how can we measure that?
- Users can better understand why their project isn't scanned by Dependency Scanning.
- Developers save a significant amount of time when adding support for a new package manager.
What is the type of buyer?
Is this a cross-stage feature?
Potentially. This could be ported to other devopssecure scanners.