Remove rules:exists from Dependency Scanning jobs

Release notes

Problem to solve

Dependency Scanning jobs are skipped when the contents of the git repo being scanned doesn't match their rules:exists parameter but are downsides to this optimization:

  • It makes it impossible to notify users about what's not supported at the moment, and link to user docs and/or the relevant featureaddition issue. (This could be done in the Rails backend but would create maintenance issues.)
  • UX is inconsistent because sometimes scanning jobs are triggered even though there's nothing to scan.
    • The files that match rules:exists are excluded by the DS_EXCLUDED_PATHS variables.
    • GitLab matches a maximum of 10,000 exists patterns or file paths, and the job is triggered regardless of what's in the repo.
  • It makes it harder to report a bug about a project not being scanned, because the scanning job is simply missing. It would be easier if users or support engineers could link to the job where the scan isn't happening.
  • It creates maintenance issues because the Dependency Scanning CI template and and the corresponding QA template must be updated when the analyzer supports new file, resulting in a change that spreads over 3 repositories, and that is really hard to coordinate.
  • It creates a gap between job integration tests and image integration tests because the latter don't cover rules:exists. It forces developers to maintain expensive job integration tests.

Running scanning jobs that aren't compatible with a GitLab project is potentially a waste of bandwidth when the Docker image of the analyzer is quite big and there's no cache. That said, users can still optimize their pipelines and skip analyzers using DS_EXCLUDED_ANALYZERS.

Proposal

Remove rules:exists parameters from the Dependency Scanning jobs:

Possible follow-up issues:

  • Write an info or warning message to the log, to suggest users they add the scanner name to DS_EXCLUDED_ANALYZERS.
  • Update DS_EXCLUDED_ANALYZERS automatically in order to skip the job. The problem is, if users use another language or package manager in their project, then it won't be scanned automatically.

Counter-proposal: change the rules:exists param to make it more generic, so that it doesn't change much, and so that it catches unsupported package managers. For instance, change the rules:exists of gemnasium-maven to match Java and Scala files, instead of POM XML, Sbt build files, and Gradle build files.

Intended users

TBD

User experience goal

As a user having a Java or Python project, when my project is not scanned by GitLab Dependency Scanning, I go to the log of the gemnasium-maven or gemnasium-python job to learn why it's not supported.

Permissions and Security

No change

Documentation

The new behavior must be documented in Dependency Scanning documentation.

Availability & Testing

  • New tests ensures that scanning jobs are triggered even when the files aren't supported

Also, it becomes possible to refactor some of the existing tests:

  • Some of the job integration tests can be converted to image integration tests; see #330093 (comment 585331906)
  • Existing job integrations tests can be implemented using child pipelines, and external test projects can be replaced with fixtures that we keep in the git repo of the analyzer project; see #297361 (comment 607755557)

What does success look like, and how can we measure that?

  • Users can better understand why their project isn't scanned by Dependency Scanning.
  • Developers save a significant amount of time when adding support for a new package manager.

What is the type of buyer?

GitLab Ultimate

Is this a cross-stage feature?

Potentially. This could be ported to other devopssecure scanners.

/cc @gonzoyumo @ifrenkel @NicoleSchwartz

Edited by Olivier Gonzalez