Ensure keys match when comparing reports
What does this MR do?
This MR adds a new compare_report_keys()
function to the compare_reports.sh
script, which ensures that the keys in both the actual and expected reports match. This check is important because fields which contain information that can vary, such as timestamps, are stripped from both reports, so without having a check to ensure the keys are the same, it's possible for one report to be completely missing a field.
This MR also adds a new test-compare-reports-script
job which uses rspec to test the logic of the compare_reports.sh
script
This has been reverted in !189 (merged). See discussion
What are the relevant issue numbers?
gitlab-org/gitlab#244829 (closed)
Testing
-
New unit tests have been added using rspec -
Manual test has been performed here to show that if a field is missing, an error is generated in the QA stage: $ sh ./compare_reports.sh cs $REPORT qa/expect/$REPORT
--- qa/expect/gl-container-scanning-report-keys-only.json +++ gl-container-scanning-report-keys-only.json @@ -61,8 +61,9 @@ }, "version": "" }, - "end_time": "", "type": "", + "start_time": "", + "end_time": "", "status": "" } }
Actual report gl-container-scanning-report.json differs from expected report qa/expect/gl-container-scanning-report.json: keys do not match
-
Manual test to ensure existing QA tests still function as expected here