Skip to content

Add Dependency Scanning template version 2

Igor Frenkelrequested to merge
ifrenkel/546404-versioned-ds-template into master

What does this MR do and why?

Add version v2 of the Dependency Scanning CI/CD template. This template removes the legacy gemnasium analyzer's CI jobs and uses only the dependency-scanning analyzer. It also uses spec:inputs to configure analyzer behaviour. Note there is still a preference for using variables to have backwards compatibility with users configuring the template via CI variables at the instance, group, and project level. So if the user specifies a variable, it will be preferred to spec:inputs or configured defaults.

This new CI/CD template allows groupcomposition analysis to deliver the new SBOM based DS analyzer without risking any breaking change on existing customers who use the v1 template (stable and latest editions). As we plan to release the new DS experience in Limited Availability in 18.5, users will start to adopt this template on gitlab.com only. There is currently no automated migration of existing users, so there is no huge risk in terms of user impact.

This also sets the path forward for other AST feature to adopt versioning of CI/CD templates, while awaiting for CI/CD components to become mature enough to support our needs. See Add versioning and spec:inputs to AST CI/CD tem... (#523986) for more details.

References

Add a new CI template to enable Dependency Scan... (#546404) • Igor Frenkel • 18.6

Details

Variable fallbacks

In order to have a smoother transition with user configs (especially Scan Execution Policies), the v2 template adds backward compatibility support for configuration variables from the v1 version of the template.

These variables are:

  • DS_PIPCOMPILE_REQUIREMENTS_FILE_NAME_PATTERN
  • DS_MAX_DEPTH
  • DS_EXCLUDED_PATHS
  • DS_INCLUDE_DEV_DEPENDENCIES
  • DS_STATIC_REACHABILITY_ENABLED
  • SECURE_LOG_LEVEL

These variables are preferred if set, otherwise the spec:inputs value is used.

In addition, 3 more variables are added. These were not in latest template and control the vulnerability scanning API functionality.

  • DS_ENABLE_VULNERABILITY_SCAN
  • DS_API_TIMEOUT
  • DS_API_SCAN_DOWNLOAD_DELAY

These variables are not part of the v1 template. They are part of the SBOM Scan API feature. However, because Scan Execution Policies do not support spec:inputs at this time, configuring these from SEP will only be possible through the variables.

Handling of $SECURE_ANALYZERS_PREFIX

$SECURE_ANALYZERS_PREFIX is used across AST templates to allow users to configure a;ll Secure analyzer behaviour at once (e.g. setting the registry URL for SAST, DAST, DS analyzers at once). Thus it is important to provide a consistent UX with other v1 templates

The v2 template keeps this support alongside configuration coming from spec:inputs. Because this variable is used to define the job image we can't use the same method of setting its value as is done with the variable fallbacks.

Handling of MR pipelines

$AST_ENABLE_MR_PIPELNES is another configuration option used across AST templates and thus has to have special handling in the v2 template for backward compatibility. However, we now default to using MR pipelines if this variable is not specified, to incentivize the usage of MR pipelines (like we did in the latest CI templates). This might cause the known duplicate pipeline issue for projects that have not specified worfklow rules or not set $AST_ENABLE_MR_PIPELINES. They can get the expected behavior by setting $AST_ENABLE_MR_PIPELINES accordingly, as documented today. In the future (V3?) we might remove these rules and simply defer to the pipeline's workflow rules but this might require a coordinated update across all AST CI/CD templates.

Acceptance testing

Testing variable fallbacks

Test fallback behaviour cannot be tested with unit tests because the variables must be set in the script section. A test script is used to test each variable for the following 3 scenarios.

  • Value from specified global variable
  • Value from specified input
  • Value from input default

This makes 3 tests for each variable fallback in the script section.

(The test works by overriding the analyzer image in the test project with an image that prints the value of a given environment variable. In this way the script section and template can be tested without modification.)

Testing template: https://gitlab.com/gitlab-org/gitlab/-/raw/e74d4d98bdb31208afeebb20c69018275bdcaf3f/lib/gitlab/ci/templates/Jobs/Dependency-Scanning.v2.gitlab-ci.yml

Test Name Value from global variable Value from input Value from default
DS_API_SCAN_DOWNLOAD_DELAY ✅ (test job, test source) ✅ (test job, test source) ✅ (test job, test source)
DS_API_TIMEOUT ✅ (test job, test source) ✅ (test job, test source) ✅ (test job, test source)
DS_ENABLE_VULNERABILITY_SCAN ✅ (test job, test source) ✅ (test job, test source) ✅ (test job, test source)
SECURE_LOG_LEVEL ✅ (test job, test source) ✅ (test job, test source) ✅ (test job, test source)
DS_STATIC_REACHABILITY_ENABLED ✅ (test job, test source) ✅ (test job, test source) ✅ (test job, test source)
DS_INCLUDE_DEV_DEPENDENCIES ✅ (test job, test source) ✅ (test job, test source) ✅ (test job, test source)
DS_EXCLUDED_PATHS ✅ (test job, test source) ✅ (test job, test source) ✅ (test job, test source)
DS_MAX_DEPTH ✅ (test job, test source) ✅ (test job, test source) ✅ (test job, test source)
DS_PIPCOMPILE_REQUIREMENTS_FILE_NAME_PATTERN ✅ (test job, test source) ✅ (test job, test source) ✅ (test job, test source)

Testing correct analyzer invocation for different project types

The following tests initiate a pipeline using the v2 template defined in this MR and check whether a Dependency Scanning job was initiated and whether it was successful.

Testing template: https://gitlab.com/gitlab-org/gitlab/-/raw/e74d4d98bdb31208afeebb20c69018275bdcaf3f/lib/gitlab/ci/templates/Jobs/Dependency-Scanning.v2.gitlab-ci.yml

Project (MR Link) Pipeline DS Job Exists DS Job Status
c-conan #2083998394 ✅ ✅
csharp-nuget-dotnetcore #2083998526 ✅ ✅
go-modules #2083998557 ✅ ✅
java-gradle #2083998705 ✅ ✅
java-maven #2083998758 ✅ ✅
js-npm #2083998846 ✅ ✅
js-pnpm #2083998954 ✅ ✅
js-yarn #2083999084 ✅ ✅
js-yarnV2 #2083999192 ✅ ✅
js-yarnV3 #2083999336 ✅ ✅
php-composer #2083999408 ✅ ✅
python-pip #2083999529 ✅ ✅
python-poetry #2083999716 ✅ ✅
ruby-bundler #2083999757 ✅ ✅
scala-sbt #2083999817 ✅ ✅
swift-cocoapods-ios #2083999927 ✅ ✅
swift-spm-serverside #2084000002 ✅ ✅

Testing Merge Request pipeline behaviour

As mentioned above, the v2 template aims to imitate default behaviour of the latest template for all AST templates. The default behaviour is to have MR pipelines enabled, so the following tests validate this behaviour for 3 types of user workflow rules including the security template:

  1. no workflow rules defined
  2. MR pipeline-only
  3. MR or branch pipeline but mutually exclusive

Additionally, (1) the test runs for the AST_ENABLE_MR_PIPELINES variable either not set (with default being true or null) or set to false. And (2) either Dependency Scanning v2 template is tested by itself or together with SAST latest to demonstrate how it behaves by itself and another AST template.

Results are shown for default, feature, and MR pipelines.

Testing template: https://gitlab.com/gitlab-org/gitlab/-/raw/e74d4d98bdb31208afeebb20c69018275bdcaf3f/lib/gitlab/ci/templates/Jobs/Dependency-Scanning.v2.gitlab-ci.yml

User config type AST_ENABLE_MR_PIPELINES value Included templates Default branch result Feature branch result Merge request branch result
No workflow rules defined AST_ENABLE_MR_PIPELINES not set DS.v2 1d54a (1 pipeline, 2 jobs) 608a1 (1 pipeline, 2 jobs) 16d3b (1 pipeline, 1 job, 1 pipeline, 1 job)
No workflow rules defined AST_ENABLE_MR_PIPELINES not set DS.v2 and SAST.latest 725bb (1 pipeline, 3 jobs) 70e7f (1 pipeline, 3 jobs) 396ac (1 pipeline, 2 jobs, 1 pipeline, 1 job)
No workflow rules defined AST_ENABLE_MR_PIPELINES set to false DS.v2 1e74c (1 pipeline, 2 jobs) 6f7ff (1 pipeline, 2 jobs) d7537 (1 pipeline, 2 jobs)
No workflow rules defined AST_ENABLE_MR_PIPELINES set to false DS.v2 and SAST.latest no pipelines b60a4 (1 pipeline, 3 jobs) 28646 (1 pipeline, 3 jobs)
MR pipelines and default branch only AST_ENABLE_MR_PIPELINES not set DS.v2 99795 (1 pipeline, 2 jobs) no pipelines f0c80 (1 pipeline, 2 jobs)
MR pipelines and default branch only AST_ENABLE_MR_PIPELINES not set DS.v2 and SAST.latest 064a8 (1 pipeline, 3 jobs) no pipelines e4fbc (1 pipeline, 3 jobs)
MR pipelines and default branch only AST_ENABLE_MR_PIPELINES set to false DS.v2 b0fc7 (1 pipeline, 2 jobs) no pipelines e878f (1 pipeline, 1 job)
MR pipelines and default branch only AST_ENABLE_MR_PIPELINES set to false DS.v2 and SAST.latest 3e05f (1 pipeline, 3 jobs) no pipelines bd5ec (1 pipeline, 1 job)
Either MR or branch pipeline but not both AST_ENABLE_MR_PIPELINES not set DS.v2 545c5 (1 pipeline, 2 jobs) 59ff9 (1 pipeline, 2 jobs) e7933 (1 pipeline, 2 jobs)
Either MR or branch pipeline but not both AST_ENABLE_MR_PIPELINES not set DS.v2 and SAST.latest 42df0 (1 pipeline, 3 jobs) 41966 (1 pipeline, 3 jobs) 4dbb9 (1 pipeline, 3 jobs)
Either MR or branch pipeline but not both AST_ENABLE_MR_PIPELINES set to false DS.v2 9e63f (1 pipeline, 2 jobs) d1620 (1 pipeline, 2 jobs) f54a7 (1 pipeline, 1 job)
Either MR or branch pipeline but not both AST_ENABLE_MR_PIPELINES set to false DS.v2 and SAST.latest 2f1c2 (1 pipeline, 3 jobs) ea266 (1 pipeline, 3 jobs) 37a64 (1 pipeline, 1 job)

How to set up and validate locally

As in acceptance tests above create a project that can currently be scanned by Dependency Scanning and include the templates from this MR.

  1. Create a test project.

  2. Include template from this MR

    include:
  - remote: https://gitlab.com/gitlab-org/gitlab/-/raw/ifrenkel/546404-versioned-ds-template/lib/gitlab/ci/templates/Jobs/Dependency-Scanning.v2.gitlab-ci.yml

debug:
  image: busybox
  script:
    - echo env

  3. Create commit and verify pipeline correctly generated.

MR acceptance checklist

Following the Development guide for GitLab CI/CD templates.

Note: corresponding ci/cd component exists under https://gitlab.com/components/dependency-scanning.

Edited by Igor Frenkel

Merge request reports