Configure Dependency Scanning analyzers to trigger pipelines of test projects
Problem to solve
In order to fully test the Dependency Scanning analyzers, we should configure multi-project pipelines and make the pipeline of analyzer projects trigger the pipelines of the related test projects, to do full Q&A.
We currently rely on either manual testing or analyzers' included test/fixtures
to ensure there are no regressions. This is time consuming and not very efficient.
Until https://gitlab.com/gitlab-org/gitlab-ee/issues/11238 is implemented we must use a workaround to ensure we are accurately checking downstream projects. Otherwise, downstreams are ignored. This will involve using a simple loop within our .gitlab-ci.yml
configuration that triggers the job and waits until the job has reported as finished (either a success or failure).
Another problem with our current fixtures
is that analyzers work off the first compatible project identifier, so a project with 2 separate package.json
files will only be scanned once (the first one found) when an analyzer works on the repo directly. This can be shown with the recent bug https://gitlab.com/gitlab-org/gitlab-ee/issues/10564, where the workaround is to rely on a test.sh
integration test to run multiple scans.
Desired behavior
Intended users
Further details
Proposal
- Define
qa
template job withinanalyzer.yml
- Within template job define helper functions:
trigger_job
andwait_for_job_to_be_done
(see inspiration) - Within template job export the
IMAGE_TAG
to allow analyzers to run with predefined image; i.e.SAST_ANALYZER_IMAGES=$CI_REGISTRY_IMAGE SAST_ANALYZER_IMAGE_TAG=$CI_COMMIT_REF_NAME
- Setup analyzer project to use
qa
template jobs to trigger downstream pipelines and pass ONLY IF downstreams are successful
Example yaml:
variables:
REPORT_FILENAME: gl-dependency-scanning-report.json
MAJOR: 2
stages:
- go
- build
- qa
- deploy
include:
- https://gitlab.com/gitlab-org/security-products/ci-templates/raw/master/includes-dev/analyzer.yml
qa_trigger java_maven:
extends: .qa_trigger
variables:
QA_PROJECT: "java-maven"
SAST_DISABLED: "true"
qa_check java_maven:
extends: .qa_check
variables:
QA_PROJECT: "java-maven"
Permissions and Security
N/A
Documentation
Any workflow and QA changes should be properly documented within QA process: https://gitlab.com/gitlab-org/security-products/release/blob/master/docs/qa_process.md
What does success look like, and how can we measure that?
Changes to a wrapper project (i.e. ~sast) will automatically trigger downstream pipelines for all relevant test projects. This will allow us to ensure our wrapper projects have the expected behavior across a variety of configurations without requiring manual testing.