Replace c and go downstreams with integration-test project
What does this MR do?
Replaces downstream c
and go
pipelines with integration-test.
This MR is an example of how we can utilize https://gitlab.com/gitlab-org/security-products/analyzers/integration-test to reduce our reliance on downstreams
See analyzers/integration-test
README for more details on the project itself.
Usage
# if using https://gitlab.com/gitlab-org/secure/tools/analyzer-scripts
docker build -t semgrep:use-integration-tests .
docker run -it --rm -v "$PWD:$PWD" -w "$PWD" \
-e TMP_IMAGE=semgrep:use-integration-tests \
-v /var/run/docker.sock:/var/run/docker.sock \
registry.gitlab.com/gitlab-org/security-products/analyzers/integration-test:stable rspec -f d
Pros
- Composability and direct control over test cases without relying on separate brittle
compare_reports.sh
script - See
GITLAB_FEATURES
custom ruleset specs for clear usecase of feature isolation we cannot currently perform via downstreams) - Can be run locally without downstreams
- (related to above) can be run by community contributors without hitting "no permissions to run downstreams" errors
- Can be parallelized or ran as segmented subset
Cons
- Requires
rspec
and familiarization with ruby testing framework. [Here's our companywide rspec best practices guidance and Odin project appears to have a pretty good tutorial.
What are the relevant issue numbers?
gitlab-org/gitlab#336821 (closed)
Does this MR meet the acceptance criteria?
-
Changelog entry added -
Tests added for this feature/bug -
Job definition updated, if necessary -
Conforms to the code review guidelines -
Conforms to the Go guidelines -
Security reports checked/validated by reviewer
Edited by Lucas Charles