Skip to content

Refactor integration tests to be able to easily extend them

Alan (Maciej) Paruszewski requested to merge refactor-integration-tests into master

Why is this change being made?

We want to add more integration tests to make sure we are supporting many images, initially I wanted to do it traditionally (see: https://gitlab.com/gitlab-org/security-products/analyzers/container-scanning/-/merge_requests?scope=all&state=all&search=%22Add+integration+specs+for%22 and !2664 (diffs)), by adding new Dockerfile and new spec file, but this generated more files that are copied and pasted from one image to another. This MR resolves that problem by doing small refactoring of specs to be able to provide environment variable (INTEGRATION_TEST_IMAGE) and running spec_integration_generic Rake task.

I wanted to use matrix: !2667 (diffs) feature for that, however there is a limitation that matrix cannot generate more than 50 jobs (for each image we are running integration tests 4 times), so we can support only 12 images. I believe it is better to explicitly define job name, so we can easily understand which for which image integration tests has failed.

Edited by Alan (Maciej) Paruszewski

Merge request reports