Replace job tests with image tests in gemnasium-python
Summary
Update gemnasium-python to replace job integration tests (using external test projects and running in separate downstream pipelines) with image integration tests (using rspec and all running in the same image test
job).
Implementation plan
-
Add rspec to the project:
- add a Gemfile
- create a
spec
directory - create an
spec/image_spec.rb
file - add an image test job
-
For each of the following downstream QA job integration tests in the gemnasium project:
-
python-pipenv-qa -
python-pipenv-offline-qa (not ported) -
python-pipenv-use-system-python-qa (removed) -
python-pipfile-lock-qa -
python-pip (not ported) -
python-pip-requirements-file-var-qa -
ds-excluded-paths-qa -
python-3.9-pipenv-qa -
python-3.9-pipenv-offline-qa (not ported) -
python-3.9-pipenv-use-system-python-qa (removed) -
python-3.9-pipfile-lock-qa -
python-3.9-pip-qa (not ported) -
python-3.9-pip-requirements-file-var-qa -
python-3.9-ds-excluded-paths-qa
Perform the following tasks:
- check whether the job integration test can be replaced by an rspec test
- if yes, copy the files from the test project to the
qa/fixtures
directory ofgemnasium
; the sub-directory name istest_project_name/test_case_name
, and the default test case is "main" - add an RSpec example to the
spec/image_spec.rb
file that replicates what's tested in the downstream pipeline - make sure that the new RSpec examples are executed when running the
image test
job - remove the downstream pipeline from the CI config
-
Also, we'll need to make sure to keep at least one job integration test, to check the Dependency Scanning template.
Image integration tests are implemented using the integration-test project. See gemnasium's spec/image_spec.rb.
Improvements
- Image integration tests can run locally.
- They are easier to maintained, compared to job integration tests using external test projects.
- They run faster because there's no need to spin off a new job for each test case.
Testing
The integration tests provide coverage. SET may need to refactor, replace or remove the Test Orchestrator.
Edited by Fabien Catteau