Configure Dependency Scanning analyzers to trigger pipelines of test projects

Problem to solve

In order to fully test the Dependency Scanning analyzers, we should configure multi-project pipelines and make the pipeline of analyzer projects trigger the pipelines of the related test projects, to do full Q&A.

We currently rely on either manual testing or analyzers' included test/fixtures to ensure there are no regressions. This is time consuming and not very efficient.

Until https://gitlab.com/gitlab-org/gitlab-ee/issues/11238 is implemented we must use a workaround to ensure we are accurately checking downstream projects. Otherwise, downstreams are ignored. This will involve using a simple loop within our .gitlab-ci.yml configuration that triggers the job and waits until the job has reported as finished (either a success or failure).

Another problem with our current fixtures is that analyzers work off the first compatible project identifier, so a project with 2 separate package.json files will only be scanned once (the first one found) when an analyzer works on the repo directly. This can be shown with the recent bug https://gitlab.com/gitlab-org/gitlab-ee/issues/10564, where the workaround is to rely on a test.sh integration test to run multiple scans.

Desired behavior

Intended users

Persona: Software developer

Further details

Proposal

1. Define qa template job within analyzer.yml
2. Within template job define helper functions: trigger_job and wait_for_job_to_be_done (see inspiration)
3. Within template job export the IMAGE_TAG to allow analyzers to run with predefined image; i.e. SAST_ANALYZER_IMAGES=$CI_REGISTRY_IMAGE SAST_ANALYZER_IMAGE_TAG=$CI_COMMIT_REF_NAME
4. Setup analyzer project to use qa template jobs to trigger downstream pipelines and pass ONLY IF downstreams are successful

Example yaml:

variables:
  REPORT_FILENAME: gl-dependency-scanning-report.json
  MAJOR: 2

stages:
  - go
  - build
  - qa
  - deploy

include:
  - https://gitlab.com/gitlab-org/security-products/ci-templates/raw/master/includes-dev/analyzer.yml

qa_trigger java_maven:
  extends: .qa_trigger
  variables:
    QA_PROJECT: "java-maven"
    SAST_DISABLED: "true"

qa_check java_maven:
  extends: .qa_check
  variables:
    QA_PROJECT: "java-maven"

Permissions and Security

N/A

Documentation

Any workflow and QA changes should be properly documented within QA process: https://gitlab.com/gitlab-org/security-products/release/blob/master/docs/qa_process.md

What does success look like, and how can we measure that?

Changes to a wrapper project (i.e. ~sast) will automatically trigger downstream pipelines for all relevant test projects. This will allow us to ensure our wrapper projects have the expected behavior across a variety of configurations without requiring manual testing.

Links / references