Skip to content

Replace c and go downstreams with integration-test project

Lucas Charles requested to merge use-integration-tests into main

What does this MR do?

Replaces downstream c and go pipelines with integration-test.

This MR is an example of how we can utilize https://gitlab.com/gitlab-org/security-products/analyzers/integration-test to reduce our reliance on downstreams

See analyzers/integration-test README for more details on the project itself.

Usage

# if using https://gitlab.com/gitlab-org/secure/tools/analyzer-scripts
docker build -t semgrep:use-integration-tests .

docker run -it --rm -v "$PWD:$PWD" -w "$PWD" \
  -e TMP_IMAGE=semgrep:use-integration-tests \
  -v /var/run/docker.sock:/var/run/docker.sock \
  registry.gitlab.com/gitlab-org/security-products/analyzers/integration-test:stable rspec -f d

Pros

  1. Composability and direct control over test cases without relying on separate brittle compare_reports.sh script
  2. See GITLAB_FEATURES custom ruleset specs for clear usecase of feature isolation we cannot currently perform via downstreams)
  3. Can be run locally without downstreams
  4. (related to above) can be run by community contributors without hitting "no permissions to run downstreams" errors
  5. Can be parallelized or ran as segmented subset

Cons

  1. Requires rspec and familiarization with ruby testing framework. [Here's our companywide rspec best practices guidance and Odin project appears to have a pretty good tutorial.

What are the relevant issue numbers?

gitlab-org/gitlab#336821 (closed)

Does this MR meet the acceptance criteria?

Edited by Lucas Charles

Merge request reports