Replace 'report with scanned files' shared example with 'recorded report' example in dependency scanning image tests
Proposal
As discussed here, we should remove the report with scanned files shared examples from the integration-test project and use the recorded report shared example in the image integration tests instead.
The report with scanned files
shared example is problematic for the following reasons:
- It assumes that all the dependency files being scanned have vulnerabilities. This requires additional work to fix, which is described in Update shared examples of integration test proj... (#354079 - closed)
- It often duplicates existing tests, like this example which duplicates this test
- When it doesn't duplicate existing tests, it still leads to additional code such as this test
- It's confusing - we can obtain the necessary coverage by using
recorded report
, which will compare the actual report with an expected report.
Implementation Plan
-
Remove report with scanned files shared examples from the integration-test project -
Replace all occurrences of it_behaves_like "report with scanned files"
withit_behaves_like "recorded report"
in the following image specs:-
gemnasium (16 occurrences) -
gemnasium-python (7 occurrences)
-
-
Replace the following workarounds and extra tests with
it_behaves_like "recorded report"
:-
gemnasium -
https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/blob/71b593e/spec/image_spec.rb#L76-85 -
https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/blob/71b593e/spec/image_spec.rb#L118-129 -
https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/blob/71b593e/spec/image_spec.rb#L447-460 -
https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/blob/71b593e/spec/image_spec.rb#L495-509
-
-
gemnasium-maven
-
Edited by Adam Cohen