Speed up merge request metrics update on deploy
What does this MR do and why?
Deployment#update_merge_request_metrics! runs on every production deployment
via Deployments::UpdateEnvironmentWorker. The query filtered only
merge_requests.target_project_id, so Postgres had no predicate on
merge_request_metrics.target_project_id and could not use any
(target_project_id, merged_at) index on that table. It drove the query from
merge_requests and probed merge_request_metrics once per MR in the project,
which is slow on large projects (observed ~31s in production slow logs).
This switches to the existing MergeRequest.join_metrics(project.id) scope,
which adds a merge_request_metrics.target_project_id = project.id predicate.
The column is denormalized, NOT NULL (check_e03d0900bf), and kept equal to
merge_requests.target_project_id on every MR save (MergeRequest::Metrics.record!),
so the result set is unchanged. With the predicate, the planner seeks
index_mr_metrics_on_target_project_id_merged_at_nulls_last directly.
Related to #628635 (closed).
SQL before and after
Generated with rails runner. Only the merge_request_metrics.target_project_id
predicate is added; everything else is identical.
Before:
UPDATE merge_request_metrics SET first_deployed_to_production_at = $1
WHERE merge_request_id IN (
SELECT merge_requests.id FROM merge_requests
INNER JOIN merge_request_metrics ON merge_request_metrics.merge_request_id = merge_requests.id
WHERE merge_requests.target_project_id = $2
AND merge_requests.target_branch = $3
AND merge_request_metrics.first_deployed_to_production_at IS NULL
AND merge_request_metrics.merged_at <= $4
AND merge_request_metrics.merged_at >= $5
) AND merge_request_metrics.first_deployed_to_production_at IS NULLAfter:
UPDATE merge_request_metrics SET first_deployed_to_production_at = $1
WHERE merge_request_id IN (
SELECT merge_requests.id FROM merge_requests
INNER JOIN merge_request_metrics ON merge_request_metrics.merge_request_id = merge_requests.id
WHERE merge_requests.target_project_id = $2
AND merge_request_metrics.target_project_id = $2
AND merge_requests.target_branch = $3
AND merge_request_metrics.first_deployed_to_production_at IS NULL
AND merge_request_metrics.merged_at <= $4
AND merge_request_metrics.merged_at >= $5
) AND merge_request_metrics.first_deployed_to_production_at IS NULLQuery plans (Database Lab)
Before, ~31s: https://postgres.ai/console/gitlab/gitlab-production-main/sessions/58244/commands/162915
-> Index Scan using index_on_merge_requests_for_latest_diffs on merge_requests
(actual time=4.870..11775.790 rows=7866 loops=1)
Index Cond: (target_project_id = 37796001)
Filter: (target_branch = 'main')
-> Index Scan using unique_merge_request_metrics_by_merge_request_id on merge_request_metrics
(actual time=2.481..2.481 rows=0 loops=7866)
Filter: (first_deployed_to_production_at IS NULL AND merged_at BETWEEN ...)
Execution time: 31.302 sAfter, ~5ms: https://postgres.ai/console/gitlab/gitlab-production-main/sessions/58244/commands/162917
-> Index Scan using index_mr_metrics_on_target_project_id_merged_at_nulls_last on merge_request_metrics
(actual time=4.771..4.771 rows=0 loops=1)
Index Cond: (target_project_id = 37796001 AND merged_at BETWEEN ...)
Filter: (first_deployed_to_production_at IS NULL)
-> Index Scan using merge_requests_pkey on merge_requests (never executed)
Execution time: 4.917 msThe pre-existing index_mr_metrics_on_target_project_id_merged_at_nulls_last
serves the query; the partial index prepared in
!254915 (merged) was not needed and
its synchronous follow-up may be droppable.