Skip to content

Resolve cross-join in ProjectsGrade.grades_for with groups

What does this MR do and why?

Remove cross-database join from Vulnerabilities::ProjectsGrade.grades_for when vulnerable objects are Group models.

This is under a feature flag and can only be tested in the context of groups, not instances.

  • Introduce new class method .grades_for_vulnerables_no_cross_join.

    This method behaves like .grades_for but doesn't have a cross-database join. It iterates batches of sub-groups and sub-batches of projects IDs to collect vulnerability_statistics.

  • Introduce remove_cross_join_from_vulnerabilities_projects_grade flag.

    .grades_for delegates to .grades_for_vulnerables_no_cross_join when all the vulnerable objects it receives are groups where the flag is enabled. It doesn't check whether the feature flag is enabled for sub-groups.

  • Add a spec for when .grades_for receives multiple groups.

    This spec ensures that the behavior doesn't change even though this is not the desired behavior. See ProjectsGrade.grades_for returns same grades fo... (#507992 - closed).

  • Update two unrelated specs to make Rubocop happy.

Please note that .grades_for might also receive an InstanceSecurityDashboard but this MR doesn't handle that case, for the following reasons:

Next steps

  1. Enable the feature flag for groups we own on gitlab.com (like gitlab-org), and make sure that new implementation (that removes cross-database joins) performs well.
  2. If it does, then roll out and remove the feature flag.
  3. Implement a similar change for the InstanceSecurityDashboard.

References

Please include cross links to any resources that are relevant to this MR This will give reviewers and future readers helpful context to give an efficient review of the changes introduced.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

How to set up and validate locally

bundle exec rspec ee/spec/models/vulnerabilities/projects_grade_spec.rb
  • When sec is shared, all specs pass.
  • When sec is separate, specs for .grades_for fail for instances, or for groups when the flag is disabled.
rspec ./ee/spec/models/vulnerabilities/projects_grade_spec.rb:60 # Vulnerabilities::ProjectsGrade.grades_for when the given vulnerable is a Group when subgroups are not included when the filter is not given when remove_cross_join_from_vulnerabilities_projects_grade is disabled returns the letter grades for given vulnerable
rspec ./ee/spec/models/vulnerabilities/projects_grade_spec.rb:85 # Vulnerabilities::ProjectsGrade.grades_for when the given vulnerable is a Group when subgroups are not included when the filter is given when remove_cross_join_from_vulnerabilities_projects_grade is disabled returns the filtered letter grade for given vulnerable
rspec ./ee/spec/models/vulnerabilities/projects_grade_spec.rb:117 # Vulnerabilities::ProjectsGrade.grades_for when the given vulnerable is a Group when subgroups are included when the filter is not given when remove_cross_join_from_vulnerabilities_projects_grade is disabled returns the letter grades for given vulnerable
rspec ./ee/spec/models/vulnerabilities/projects_grade_spec.rb:136 # Vulnerabilities::ProjectsGrade.grades_for when the given vulnerable is a Group when subgroups are included when the filter is not given when multiple batches are required when remove_cross_join_from_vulnerabilities_projects_grade is disabled returns the letter grades for given vulnerable
rspec ./ee/spec/models/vulnerabilities/projects_grade_spec.rb:162 # Vulnerabilities::ProjectsGrade.grades_for when the given vulnerable is a Group when subgroups are included when the filter is given when remove_cross_join_from_vulnerabilities_projects_grade is disabled returns the filtered letter grade for given vulnerable
rspec ./ee/spec/models/vulnerabilities/projects_grade_spec.rb:196 # Vulnerabilities::ProjectsGrade.grades_for when the given vulnerables are groups when remove_cross_join_from_vulnerabilities_projects_grade is disabled in one of the groups returns all letter grades for each vulnerable
rspec ./ee/spec/models/vulnerabilities/projects_grade_spec.rb:227 # Vulnerabilities::ProjectsGrade.grades_for when the given vulnerable is an InstanceSecurityDashboard when the filter is not given returns the letter grades for given vulnerable
rspec ./ee/spec/models/vulnerabilities/projects_grade_spec.rb:242 # Vulnerabilities::ProjectsGrade.grades_for when the given vulnerable is an InstanceSecurityDashboard when the filter is given returns the filtered letter grade for given vulnerable

Related to #503387 (closed)

Edited by Fabien Catteau

Merge request reports

Loading