Replace downstream pipelines with child pipelines in Dependency Scanning analyzer projects
Summary
Change job integration tests so that they use fixtures
directory of the analyzer projects and child pipelines instead of external tests projects and downstream pipelines. See #297361 (comment 490022538)
This is done after changing most job tests (1 pipeline per test case) to image tests (1 job for all test cases). See #336322 (closed)
Proposal
For each downstream pipeline using a branch of a Secure test project:
- copy files from the test project to the
qa/fixtures
directory of the analyzer project; the sub-directory name istest_project_name/test_case_name
, and the default test case is "main" - add a CI config to
qa/templates
, to be used by child pipelines - update the CI config of the analyzer to change downstream pipelines using external project to child pipelines using fixtures directory
See PoC.
The child pipeline job includes the CI config specific to job integration tests using trigger:include
, and it passes FIXTURES_DIR
path or equivalent.
php-composer-qa:
variables:
DS_ANALYZER_IMAGE: "$CI_REGISTRY_IMAGE/tmp:$CI_COMMIT_SHA"
FIXTURES_DIR: "qa/fixtures/php-composer"
DS_REPORT_URL: "$CI_PROJECT_URL/raw/$CI_COMMIT_REF_NAME/qa/expect/php-composer/$REPORT_FILENAME"
MAX_SCAN_DURATION_SECONDS: 3
trigger:
include: qa/templates/.gitlab-ci.yml
The CI config overrides the CI_PROJECT_DIR
to enter the FIXTURES_DIR
before the scan, and to copy the generated report from that directory after the scan.
stages:
- test
- qa
include:
- template: Dependency-Scanning.gitlab-ci.yml
- https://gitlab.com/gitlab-org/security-products/ci-templates/raw/master/includes-dev/qa-dependency_scanning.yml
dependency_scanning:
before_script:
- export CI_PROJECT_DIR="$CI_PROJECT_DIR/$FIXTURES_DIR"
after_script:
- cp "$CI_PROJECT_DIR/$FIXTURES_DIR/gl-dependency-scanning-report.json" "$CI_PROJECT_DIR/"
Implementation plan
Repeat the steps of the proposal for all Dependency Scanning analyzer projects:
- https://gitlab.com/gitlab-org/security-products/analyzers/bundler-audit
- https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium
- https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven
- https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python
- https://gitlab.com/gitlab-org/security-products/analyzers/retire.js
Improvements
Job integration tests can run in any fork of the analyzer projects, including forks from community contributors.
Since test projects live in the fixtures
directory of the analyzer project, they can be reviewed along with the code, in the same MR.
Risks
Child pipelines might inherit CI variables from the analyzer pipeline, resulting in differences between the test environment and normal production environment when running scanning jobs. This shouldn't be a problem (comment).
Developers adding support for new dependency files might forget that the rules:exists
parameter needs to be updated in the CI template and the corresponding spec file.
Involved components
- https://gitlab.com/gitlab-org/security-products/analyzers/bundler-audit
- https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium
- https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven
- https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python
- https://gitlab.com/gitlab-org/security-products/analyzers/retire.js
Optional: Intended side effects
See improvements
Optional: Missing test coverage
See risks