Speed up merge request metrics update on deploy

What does this MR do and why?

Deployment#update_merge_request_metrics! runs on every production deployment via Deployments::UpdateEnvironmentWorker. The query filtered only merge_requests.target_project_id, so Postgres had no predicate on merge_request_metrics.target_project_id and could not use any (target_project_id, merged_at) index on that table. It drove the query from merge_requests and probed merge_request_metrics once per MR in the project, which is slow on large projects (observed ~31s in production slow logs).

This switches to the existing MergeRequest.join_metrics(project.id) scope, which adds a merge_request_metrics.target_project_id = project.id predicate. The column is denormalized, NOT NULL (check_e03d0900bf), and kept equal to merge_requests.target_project_id on every MR save (MergeRequest::Metrics.record!), so the result set is unchanged. With the predicate, the planner seeks index_mr_metrics_on_target_project_id_merged_at_nulls_last directly.

Related to #628635 (closed).

SQL before and after

Generated with rails runner. Only the merge_request_metrics.target_project_id predicate is added; everything else is identical.

Before:

UPDATE merge_request_metrics SET first_deployed_to_production_at = $1
WHERE merge_request_id IN (
  SELECT merge_requests.id FROM merge_requests
  INNER JOIN merge_request_metrics ON merge_request_metrics.merge_request_id = merge_requests.id
  WHERE merge_requests.target_project_id = $2
    AND merge_requests.target_branch = $3
    AND merge_request_metrics.first_deployed_to_production_at IS NULL
    AND merge_request_metrics.merged_at <= $4
    AND merge_request_metrics.merged_at >= $5
) AND merge_request_metrics.first_deployed_to_production_at IS NULL

After:

UPDATE merge_request_metrics SET first_deployed_to_production_at = $1
WHERE merge_request_id IN (
  SELECT merge_requests.id FROM merge_requests
  INNER JOIN merge_request_metrics ON merge_request_metrics.merge_request_id = merge_requests.id
  WHERE merge_requests.target_project_id = $2
    AND merge_request_metrics.target_project_id = $2
    AND merge_requests.target_branch = $3
    AND merge_request_metrics.first_deployed_to_production_at IS NULL
    AND merge_request_metrics.merged_at <= $4
    AND merge_request_metrics.merged_at >= $5
) AND merge_request_metrics.first_deployed_to_production_at IS NULL

Query plans (Database Lab)

Before, ~31s: https://postgres.ai/console/gitlab/gitlab-production-main/sessions/58244/commands/162915

->  Index Scan using index_on_merge_requests_for_latest_diffs on merge_requests
      (actual time=4.870..11775.790 rows=7866 loops=1)
      Index Cond: (target_project_id = 37796001)
      Filter: (target_branch = 'main')
->  Index Scan using unique_merge_request_metrics_by_merge_request_id on merge_request_metrics
      (actual time=2.481..2.481 rows=0 loops=7866)
      Filter: (first_deployed_to_production_at IS NULL AND merged_at BETWEEN ...)
Execution time: 31.302 s

After, ~5ms: https://postgres.ai/console/gitlab/gitlab-production-main/sessions/58244/commands/162917

->  Index Scan using index_mr_metrics_on_target_project_id_merged_at_nulls_last on merge_request_metrics
      (actual time=4.771..4.771 rows=0 loops=1)
      Index Cond: (target_project_id = 37796001 AND merged_at BETWEEN ...)
      Filter: (first_deployed_to_production_at IS NULL)
->  Index Scan using merge_requests_pkey on merge_requests (never executed)
Execution time: 4.917 ms

The pre-existing index_mr_metrics_on_target_project_id_merged_at_nulls_last serves the query; the partial index prepared in !254915 (merged) was not needed and its synchronous follow-up may be droppable.

Edited by Stan Hu

Merge request reports

Loading
Loading