Replace VulnerabilitiesFinder with VulnerabilityReadsFinder
What does this MR do and why?
Replace ::Security::VulnerabilitiesFinder with ::Security::VulnerabilityReadsFinder . Part 1 of the split of !171725 (closed).
Relevant discussion: !171725 (comment 2222215770)
- Relevant discussion (comments starting from !171725 (comment 2222215770))
- This was needed because it has the denormalized
archiveattribute and namespace traversal IDs to more easily query vulnerabilities. - The
::Security::VulnerabilitiesFinderhas no more use and will be deleted.
References
Please include cross links to any resources that are relevant to this MR. This will give reviewers and future readers helpful context to give an efficient review of the changes introduced.
- &14197 (closed)
- Original MR: !171725 (closed)
Query Analysis
Query plans for the new methods/scopes in this MR, the queries are obtained from calling the new methods through the Rails console. Data used was in accordance with: https://docs.gitlab.com/ee/development/database_review.html#query-plans
We skipped the vulns/projects/license selects as those are pretty straightforward constant time and were already existing, standard behaviour for the finders.
LIMIT 1 Cursor start for projects iteration: https://console.postgres.ai/gitlab/gitlab-production-main/sessions/34959/commands/108264
LIMIT 10 OFFSET 10 for second batch of projects: https://postgres.ai/console/gitlab/gitlab-production-main/sessions/35103/commands/108817
Worst Case (No Hits):
- 1st Batch 230MB 2s https://postgres.ai/console/gitlab/gitlab-production-main/sessions/35058/commands/108709
- 2nd Batch 110MB <1s https://postgres.ai/console/gitlab/gitlab-production-main/sessions/35058/commands/108710
- 3rd Batch 3.70GB 5.8s https://postgres.ai/console/gitlab/gitlab-production-main/sessions/35058/commands/108712
- 4th Batch 4.70GB 3.6s https://postgres.ai/console/gitlab/gitlab-production-main/sessions/35058/commands/108713
Best Case (5 Hits Found on first batch):
- 1st Batch 260MB 9.4s https://postgres.ai/console/gitlab/gitlab-production-main/sessions/35103/commands/108819
- Short circuit logic skips further batches.
Project batch size is set to 10 to reduce risk of statement timeout. We expect faster execution time in production, so this should be largely stable, but if there are problems, we have considered implementing a timelimit as well.
For more context on this analysis, view the conversation here.
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Related to #499694 (closed)