Replace jq in compare_reports script in ci-templates project to pure ruby
Problem to solve
We're currently using jq in the compare_reports.sh script for filtering/sorting/sanitizing JSON before comparing the actual generated report to the expected report in the QA stage. Our usage of jq has become very complex, to the point where we need to start writing tests to ensure it functions as expected.
In order to improve the maintainability and flexibility of the compare_reports.sh shell script, we will eventually be migrating it to a ruby implementation.
Once this migration to ruby has been completed, the next step is to replace our usage of jq with a pure ruby implementation, which is the purpose of this issue.
Further details
compare_reports.sh currently goes through the following steps:
-
sanitize_and_diff_reports- filter the actual and expected reports using
jqfilters; the exact filter depends on the report type - compare the actual report to the expected report using
diff -u -b - report an error and exit 1 if there's a diff
- filter the actual and expected reports using
-
validate_report_json- get the commit ref of
security-report-schemasbased on the version stored in the actual report - get the filename of the schema based on the type of the actual report
- fetch the JSON schema
- do JSON schema validation using Python
- report an error and exit 65 if validation fails
- get the commit ref of
jq filters are used in the first step, when preparing the reports before comparing them.
Intended users
User experience goal
The compare_reports.sh script will be easier to maintain and follow by other developers, and we can add more test coverage using standard ruby tools such as rspec
Implementation plan
with #299038 (closed) we'll get a ruby script that do the same job as compare_reports.sh)
- Replace all the jq filtering logic in the new compare_reports ruby script to ruby
- Write unit tests if found missing test cases
Further details
See this discussion for more details
Availability & Testing
Unit tests will be added for the filtering/sorting/sanitizing component of compare_reports.rb ruby script
Available Tier
What does success look like, and how can we measure that?
jq from the compare_reports.sh script is replaced by a ruby implementation with unit tests.
What is the type of buyer?
GitLab Ultimate Enterprise Edition
Is this a cross-stage feature?
Yes, this affects all secure projects