Skip to content

Improve the MR analytics list query

Adam Hegyi requested to merge improve-mr-analytics-graphql-count-query into master

What does this MR do?

This MR improves the MR analytics list query by rearranging the where conditions on the query.

How to reproduce it:

  1. Enable the performance bar.
  2. Open: https://gitlab.com/gitlab-org/gitlab/-/analytics/merge_request_analytics
  3. Find the slowest Graphql endpoint in the dropdown and find the first query.

Slow query:

SELECT COUNT(*)
FROM
  (SELECT 1 AS one
   FROM "merge_requests"
   INNER JOIN "merge_request_metrics" ON "merge_request_metrics"."merge_request_id" = "merge_requests"."id"
   WHERE "merge_requests"."target_project_id" = 278964
     AND "merge_requests"."target_project_id" = "merge_request_metrics"."target_project_id"
     AND "merge_request_metrics"."merged_at" >= '2020-03-11 00:00:00'
     AND "merge_request_metrics"."merged_at" <= '2021-03-11 00:00:00'
     AND "merge_requests"."target_project_id" = "merge_request_metrics"."target_project_id"
   LIMIT 21) subquery_for_count

Plan: https://explain.depesz.com/s/CxLS

Why is it slow?

The planner decides to hash join the matching merge_requests rows with the merge_request_metrics rows. After that it takes out 21 rows.

Improved query:

SELECT COUNT(*)
FROM
  (SELECT 1 AS one
   FROM "merge_requests"
   INNER JOIN "merge_request_metrics" ON "merge_request_metrics"."merge_request_id" = "merge_requests"."id"
   WHERE "merge_request_metrics"."target_project_id" = 278964
   LIMIT 21) subquery_for_count

Plan: https://explain.depesz.com/s/ekJ3

The fix identifies if the MergeRequestsFinder queries one project only. In that case the target_project_id condition is removed from the original scope and added to the joined merge_request_metrics scope. MergeRequest::Metrics has the same target_project_id column (de-normalization).

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Adam Hegyi

Merge request reports