Gate commits partition pruning on separate FF

Eugenia Grieffrequested to merge
600540-add-commits-table-pruning into master

What does this MR do and why?

Context

The new commits table is partitioned by project_id so as part of https://gitlab.com/groups/gitlab-org/-/work_items/18350 we have already updated some of the queries to include this column for partition pruning.

These changes were behind the FF mr_diff_commits_read_new_table, but need to be moved to a different flag because we want to enable the read flag before we swap tables and this would break the commits queries because project_id is not populated in the old table.

This MR's scope

Create a new FF so partition pruning can be gated independently and enabled only after the swap (which confirms all project_id rows are populated), and update remaining queries.

Changes

  • In places where the project_id filter was already added, replace the flag mr_diff_commits_read_new_table with a new one mr_diff_commits_project_id_pruning
  • Handle merge_request_diff_commits has_many association in MergeRequestDiff where commits were queried without project_id (changed added behind the same FF)
  • Implement a new RSpec matcher (query_diff_commits_without_project_id) to verify the filter is not present when FF is disabled and update existing specs to use it.

Database changes

The only change that generates a materially different query shape is the has_many :merge_request_diff_commits association, which adds WHERE merge_request_diff_commits.project_id = X to association loads. See !240394 (comment 3504207135) for query plans.

Everything else on this MR is one of:

  • Feature flag substitution (swap check read_new_commits_table? for project_id_pruning_enabled?): same SQL.
  • Cosmetically equivalent join (swap joins(:merge_request_diff_commits) for a literal SQL): The literal SQL is functionally the same join.
  • Query removal (the includes drop): removes a preload, doesn't introduce a new shape.

References

Related to #594699

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Eugenia Grieff

Merge request reports