Replace downstream pipelines with child pipelines in Dependency Scanning analyzer projects

Summary

Change job integration tests so that they use fixtures directory of the analyzer projects and child pipelines instead of external tests projects and downstream pipelines. See #297361 (comment 490022538)

This is done after changing most job tests (1 pipeline per test case) to image tests (1 job for all test cases). See #336322 (closed)

Proposal

For each downstream pipeline using a branch of a Secure test project:

  • copy files from the test project to the qa/fixtures directory of the analyzer project; the sub-directory name is test_project_name/test_case_name, and the default test case is "main"
  • add a CI config to qa/templates, to be used by child pipelines
  • update the CI config of the analyzer to change downstream pipelines using external project to child pipelines using fixtures directory

See PoC.

The child pipeline job includes the CI config specific to job integration tests using trigger:include, and it passes FIXTURES_DIR path or equivalent.

php-composer-qa:
  variables:
    DS_ANALYZER_IMAGE: "$CI_REGISTRY_IMAGE/tmp:$CI_COMMIT_SHA"
    FIXTURES_DIR: "qa/fixtures/php-composer"
    DS_REPORT_URL: "$CI_PROJECT_URL/raw/$CI_COMMIT_REF_NAME/qa/expect/php-composer/$REPORT_FILENAME"
    MAX_SCAN_DURATION_SECONDS: 3
  trigger:
    include: qa/templates/.gitlab-ci.yml

The CI config overrides the CI_PROJECT_DIR to enter the FIXTURES_DIR before the scan, and to copy the generated report from that directory after the scan.

stages:
  - test
  - qa

include:
  - template: Dependency-Scanning.gitlab-ci.yml
  - https://gitlab.com/gitlab-org/security-products/ci-templates/raw/master/includes-dev/qa-dependency_scanning.yml

dependency_scanning:
  before_script:
    - export CI_PROJECT_DIR="$CI_PROJECT_DIR/$FIXTURES_DIR"
  after_script:
    - cp "$CI_PROJECT_DIR/$FIXTURES_DIR/gl-dependency-scanning-report.json" "$CI_PROJECT_DIR/"

Implementation plan

Repeat the steps of the proposal for all Dependency Scanning analyzer projects:

Improvements

Job integration tests can run in any fork of the analyzer projects, including forks from community contributors.

Since test projects live in the fixtures directory of the analyzer project, they can be reviewed along with the code, in the same MR.

Risks

Child pipelines might inherit CI variables from the analyzer pipeline, resulting in differences between the test environment and normal production environment when running scanning jobs. This shouldn't be a problem (comment).

Developers adding support for new dependency files might forget that the rules:exists parameter needs to be updated in the CI template and the corresponding spec file.

Involved components

Optional: Intended side effects

See improvements

Optional: Missing test coverage

See risks

Edited by Fabien Catteau