Add ability to optionally ignore dev dependencies in NPM projects (Gemnasium)

Problem to solve

When running Dependency Scanning on Node.JS npm projects, Gemnasium scans all dependencies, including devDependencies. However, typically devDependencies are not installed in a production environment, and so they don't warrant security warnings. The dependencies being scanned shouldn't be the ones installed when running npm install --production or npm ci --production.

For instance, in a project having the following package.json dependency file, my_test_framework and another_dev_dep shouldn't be scanned.

{
  "dependencies": {
    "my_dep": "^1.0.0",
    "another_dep": "~2.2.0"
  },
  "devDependencies" : {
    "my_test_framework": "^3.1.0".
    "another_dev_dep": "1.0.0 - 1.2.0"
  }
}

Intended users

• Sasha (Software Developer)
• Devon (DevOps Engineer)
• Sam (Security Analyst)
• Alex (Security Operations Engineer)

Proposal

Introduce a new CI variable named DS_INCLUDE_DEV_DEPENDENCIES. When it's "false", the Dependency Scanning job ignores development dependencies when scanning npm projects.

DS_INCLUDE_DEV_DEPENDENCIES is false by default to ensure backward compatibility. The behavior doesn't change.

Further details

Technically, we can leverage the dev field of the dependencies listed in npm lock files. See https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/v2.35.0/scanner/parser/npm/fixtures/simple/package-lock.json#L11

• parser.ParseFunc is changed to accept parsing options of type parser.Options. Options.IncludeDev (bool) is set to make the parser ignore dev dependencies.
• npm.Parse is changed to capture the dev field. If parser.Options.IncludeDev is false, then it skips dependencies where dev is true. If a dependency is dev and non-dev, then it should not be skipped.
• scanner.Scanner has a new parsingOptions field. scanner.Flags and scanner.NewParser are updated so that parsing options can be configured using CLI flags. Parsing options are passed to the Parse function, in Scanner.scanFile.

Documentation

Document DS_INCLUDE_DEV_DEPENDENCIES in Configuring specific analyzers used by dependency scanning.

• It's only supported for npm projects.
• It defaults to false, and dev dependencies are not skipped by default.
• When set to "false", dev dependencies are excluded from the scan; dev dependencies and their vulnerabilities are not reported.

Testing

A new image spec (integration test) is added to gemnasium, to check that devDependencies are ignored when DS_INCLUDE_DEV_DEPENDENCIES is "false".

Also, the unit tests of the npm parser are updated, to test the extraction of the dev boolean field.

Implementation plan

• Update gemnasium
  • Pass parsing options to lock file parsers.
  • Introduce new IncludeDev parsing option.
  • Update npm lock file parser to skip dev dependencies depending on parsing options.
  • Update scanner to set up parsing options based on CLI flags.
  • Add image spec for npm project when DS_INCLUDE_DEV_DEPENDENCIES is "false".
• Update documentation
  • Document DS_INCLUDE_DEV_DEPENDENCIES.
• Create issues to cover all package managers supported by Gemnasium.