Investigate possible incremental improvements to backend API performance for Code Quality reports
Problem
Code Quality reports load more slowly as they get larger. We need to investigate if there are some options to improve this experience without spending too much effort on it.
Proposal(s) to investigate further
Store summary of report diff in the database
We could create a table to store a summary of the Code Quality jobs for a given pipeline. This table might include: pipeline_id, total_findings_count, new_findings_count, fixed_findings_count, target_branch.
This information could be used for:
- Quickly displaying a summary on the CQ pipeline tab (example), which could help with the LCP score (if the summary is enough info to count for the LCP)
- The new API needed for [MR Widget Eng] Code Quality - v2 - Use dedicated API endpoint for collapsed/uncollapsed state
- Be used to drive logic decisions in future iterations. For example, if the report has over 1,000 findings, we could display just the new findings for a given report. This may not speed up the initial display of all the results, but it should make this listing more useful as it's hard, and slow, to scroll through 1,000s of results.
Other pending questions:
- Would we need to backfill these records for previous pipelines?
- Would we need to do record cleanup as the associated reports will expire over time?
Store summary of report diff as a json artifact
Instead of creating a new table, as suggested in the above suggestion, we could add Ci::PipelineArtifact records in the DB that point to a file containing JSON containing the same info.
It may also be possible to change the JSON that we are generating for the Code Quality MR diff report so that it contains all the necessary info, which would mean we wouldn't need to generate a different artifact.
Store summary of report severity type counts
Add GraphQL support of 'CodeQualityReportSummary' is adding code to calculate counts for the various severity types for a given Code Quality report. This will be done every time CodeQualityReportSummary is loaded, and will be slower for large reports. Storing this info in the database could be a way to speed this up.