Spike: Audit DS analyzer options that impact downstream features and might need to be ported to SBOM based scans

Proposal

As part of Challenges of SBOM based Dependency Scanning (&11617 - closed) we have identified that existing options available in our current Dependency Scanning implementation (CI job analyzer) can impact the behavior of downstream features like the dependency list, dependency scanning, license scanning, etc.

In order to prepare for the transition to SBOM based scans we must audit these options and decide if they should be ported to maintain the same (or similar) capabilities for our users.

Audit

DS Analyzer ENV VARs documented in user doc

Variable	Needed for Gemnasium deprecation?	Why?
ADDITIONAL_CA_CERT_BUNDLE	No	No network connections in new analyzer
DS_EXCLUDED_ANALYZERS	No	New analyzer replaces all existing ones
DS_EXCLUDED_PATHS	Yes	Monorepo support
DS_IMAGE_SUFFIX	No	New analyzer replaces all existing ones
DS_MAX_DEPTH	Yes	Monorepo support
SECURE_ANALYZERS_PREFIX	Yes	Offline support
GEMNASIUM_DB_*	No	Offline support is built-in to CVS
DS_REMEDIATE*	No	Replaced by &759
GEMNASIUM_LIBRARY_SCAN_ENABLED	No	Vendored JavaScript libraries support
DS_INCLUDE_DEV_DEPENDENCIES	Yes	Replacement by UI filter is unlikely to be done in time; discussion
GO*	No	No build support
DS_JAVA_VERSION	No	No build support
MAVEN_CLI_OPTS	No	No build support
GRADLE_*	No	No build support
SBT_CLI_OPTS	No	No build support
PIP*	No	No build support
DS_PIP_*	No	No build support
HTTPS_PROXY	No	No build support

DS Analyzer CLI options (not documented in the user doc)

View options description

--additional-ca-cert-bundle value                  Additional CA certs bundle to import [$ADDITIONAL_CA_CERT_BUNDLE]
--artifact-dir value                               Artifact directory [$ANALYZER_ARTIFACT_DIR, $CI_PROJECT_DIR]
--excluded-paths value [ --excluded-paths value ]  Comma-separated list of paths (globs supported) to be excluded from the output. [$DS_EXCLUDED_PATHS]
--fips-mode                                        FIPS mode enabled (default: false) [$FIPS_MODE]
--gemnasium-db-local-path value                    Path of gemnasium-db git repo (default: "gemnasium-db") [$GEMNASIUM_DB_LOCAL_PATH]
--gemnasium-db-ref-name value                      git reference the local gemnasium-db git repo is synced with [$GEMNASIUM_DB_REF_NAME]
--gemnasium-db-remote-url value                    Remote URL the local gemnasium-db git repo is synced with [$GEMNASIUM_DB_REMOTE_URL]
--gemnasium-db-update-disabled                     Disable gemnasium-db git repo update before scanning (default: false) [$GEMNASIUM_DB_UPDATE_DISABLED]
--gemnasium-db-web-url value                       Web URL of the gemnasium-db GitLab project [$GEMNASIUM_DB_WEB_URL]
--get-pip-path value                               Path to the get-pip.py script (default: "get-pip.py") [$DS_GET_PIP_PATH]
--gradle-init-script value                         Optional CLI argument pointing to the gradle init script (default: "gemnasium-init.gradle") [$GRADLE_PLUGIN_INIT_PATH]
--gradle-opts value                                Optional CLI arguments for the gradle dependency dump command [$GRADLE_CLI_OPTS]
--ignore-hidden-dirs                               Ignore hidden directories (default: true) [$SEARCH_IGNORE_HIDDEN_DIRS]
--ignored-dirs value [ --ignored-dirs value ]      Directory to be ignored (default: "node_modules", ".bundle", "vendor", ".git") [$SEARCH_IGNORED_DIRS]
--include-dev-deps                                 Include development dependencies in scan (default: true) [$DS_INCLUDE_DEV_DEPENDENCIES]
--maven-opts value                                 Optional CLI arguments for the maven install command (default: "-DskipTests --batch-mode") [$MAVEN_CLI_OPTS]
--max-depth value                                  Maximum directory depth, set to -1 to ignore (default: 2) [$DS_MAX_DEPTH, $SEARCH_MAX_DEPTH]
--pip-cert-path value                              Path to certificate file to be used in pip and pipenv installs [$PIP_CERT]
--pip-dependency-path value                        Path to directory with pip dependencies [$DS_PIP_DEPENDENCY_PATH]
--pip-requirements-file value                      Custom requirements file to use when analyzing project [$PIP_REQUIREMENTS_FILE]
--pip-version value                                Pip version to install and use [$DS_PIP_VERSION]
--remediate                                        Remediate vulnerabilities (default: true) [$DS_REMEDIATE]
--remediate-timeout value                          Time limit for vulnerabilities auto-remediation (default: 5m0s) [$DS_REMEDIATE_TIMEOUT]
--sbt-opts value                                   Optional CLI arguments for sbt [$SBT_CLI_OPTS]
--scan-libs                                        Scan vendored libraries (default: false) [$GEMNASIUM_LIBRARY_SCAN_ENABLED]
--schema-model value                               Model of the Security Report JSON schema (default: 15) [$DS_SCHEMA_MODEL]
--target-dir value                                 Target directory [$ANALYZER_TARGET_DIR, $CI_PROJECT_DIR]
--vrange-cargo-cmd value                           Path to CLI command that evaluates version range for cargo (default: "npm/rangecheck.js") [$VRANGE_CARGO_CMD]
--vrange-cocoapods-cmd value                       Path to CLI command that evaluates version range for cocoapods (default: "gem/vrange.rb") [$VRANGE_COCOAPODS_CMD]
--vrange-conan-cmd value                           Path to CLI command that evaluates version range for conan (default: "npm/rangecheck.js") [$VRANGE_CONAN_CMD]
--vrange-dir value                                 Base directory of vrange scripts [$VRANGE_DIR]
--vrange-gem-cmd value                             Path to CLI command that evaluates version range for gem (default: "gem/vrange.rb") [$VRANGE_GEM_CMD]
--vrange-maven-cmd value                           Path to CLI command that evaluates version range for maven (default: "semver/vrange-linux maven") [$VRANGE_MAVEN_CMD]
--vrange-npm-cmd value                             Path to CLI command that evaluates version range for npm (default: "npm/rangecheck.js") [$VRANGE_NPM_CMD]
--vrange-nuget-cmd value                           Path to CLI command that evaluates version range for nuget (default: "nuget/vrange-linux") [$VRANGE_NUGET_CMD]
--vrange-php-cmd value                             Path to CLI command that evaluates version range for php (default: "php/rangecheck.php") [$VRANGE_PHP_CMD]
--vrange-swift-cmd value                           Path to CLI command that evaluates version range for swift (default: "npm/rangecheck.js") [$VRANGE_SWIFT_CMD]

Variable	Needed for Gemnasium deprecation?	Why?	Approved?
DS_GET_PIP_PATH	No	Internal, use for installing pip defined with DS_PIP_VERSION
SEARCH_IGNORED_DIRS	No	Replaced by DS_EXCLUDED_PATHS
SEARCH_IGNORE_HIDDEN_DIRS	Maybe	Internal, allow to alter detection logic (skip compatible files in hidden directory)
SEARCH_MAX_DEPTH (DS_MAX_DEPTH alias)	No	Allow user to alter detection logic (how deep to look for compatible files)
FIPS_MODE	Maybe	If new SBOM generator needs FIPS specific behavior
ANALYZER_TARGET_DIR, CI_PROJECT_DIR	No	Allow user to alter detection logic (where to look for compatible files)
ANALYZER_ARTIFACT_DIR, CI_PROJECT_DIR	No	Allow user to alter where report artifacts are written
VRANGE_*	No	Internal, only useful for doing DS scan in CI
DS_SCHEMA_MODEL	No	Internal, only useful for DS report generation in CI