Update secure analyzers CI to trigger blocking bridge jobs for test projects
Problem to solve
In order to properly test and QA our analyzers and their wrapper ~Secure projects we should setup multi-project pipelines to trigger downstream executions of our test projects from analyzer projects.
We currently rely on either manual testing or analyzers' included
test/fixtures to ensure there are no regressions. This is time consuming and not very efficient.
Until https://gitlab.com/gitlab-org/gitlab-ee/issues/11238 is implemented we must use a workaround to ensure we are accurately checking downstream projects. Otherwise, downstreams are ignored. This will involve using a simple loop within our
.gitlab-ci.yml configuration that triggers the job and waits until the job has reported as finished (either a success or failure).
Another problem with our current
fixtures is that analyzers work off the first compatible project identifier, so a project with 2 separate
package.json files will only be scanned once (the first one found) when an analyzer works on the repo directly. This can be shown with the recent bug https://gitlab.com/gitlab-org/gitlab-ee/issues/10564, where the workaround is to rely on a
test.sh integration test to run multiple scans.
qatemplate job within
- Within template job define helper functions:
- Within template job export the
IMAGE_TAGto allow analyzers to run with predefined image; i.e.
- Setup analyzer project to use
qatemplate jobs to trigger downstream pipelines and pass ONLY IF downstreams are successful
variables: REPORT_FILENAME: gl-dependency-scanning-report.json MAJOR: 2 stages: - go - build - qa - deploy include: - https://gitlab.com/gitlab-org/security-products/ci-templates/raw/master/includes-dev/analyzer.yml qa_trigger java_maven: extends: .qa_trigger variables: QA_PROJECT: "java-maven" SAST_DISABLED: "true" qa_check java_maven: extends: .qa_check variables: QA_PROJECT: "java-maven"
Permissions and Security
Any workflow and QA changes should be properly documented within QA process: https://gitlab.com/gitlab-org/security-products/release/blob/master/docs/qa_process.md
What does success look like, and how can we measure that?
Changes to a wrapper project (i.e. sast) will automatically trigger downstream pipelines for all relevant test projects. This will allow us to ensure our wrapper projects have the expected behavior across a variety of configurations without requiring manual testing.