Add index to improve counting of open issues
What does this MR do and why?
Adds an index to improve the performance of counting open issues of a project. This query is triggered every time an issue is created or changes state.
This MR only prepares the index for weekend creation on GitLab.com: https://docs.gitlab.com/ee/development/adding_database_indexes.html#create-indexes-asynchronously
Query:
SELECT COUNT(*) AS count_all, "issues"."confidential" AS issues_confidential
FROM "issues"
WHERE ("issues"."state_id" IN (1)) AND "issues"."project_id" = 19923279
GROUP BY "issues"."confidential"
Note: The query right now in master
includes an issue_type
filter, but that will be removed by !89118 (merged)
Before:
HashAggregate (cost=51322.17..51322.19 rows=2 width=9) (actual time=249.499..249.501 rows=2 loops=1)
Group Key: issues.confidential
Buffers: shared hit=83043
I/O Timings: read=0.000 write=0.000
-> Index Scan using index_issue_on_project_id_state_id_and_blocking_issues_count on public.issues (cost=0.56..51142.76 rows=35883 width=1) (actual time=0.047..221.147 rows=91423 loops=1)
Index Cond: ((issues.project_id = 19923279) AND (issues.state_id = 1))
Buffers: shared hit=83043
I/O Timings: read=0.000 write=0.000
Time: 253.770 ms
- planning: 4.148 ms
- execution: 249.622 ms
- I/O read: 0.000 ms
- I/O write: 0.000 ms
Shared buffers:
- hits: 83043 (~648.80 MiB) from the buffer pool
- reads: 0 from the OS file cache, including disk I/O
- dirtied: 0
- writes: 0
Sometimes the query is fast like above, but sometimes it times out as seen in the linked issue.
After:
Aggregate (cost=0.56..3454.36 rows=2 width=9) (actual time=0.243..55.344 rows=2 loops=1)
Group Key: issues.confidential
Buffers: shared hit=4931
I/O Timings: read=0.000 write=0.000
-> Index Only Scan using idx_open_issues_on_project_id_and_confidential on public.issues (cost=0.56..3274.93 rows=35882 width=1) (actual time=0.150..40.636 rows=91423 loops=1)
Index Cond: (issues.project_id = 19923279)
Heap Fetches: 4986
Buffers: shared hit=4931
I/O Timings: read=0.000 write=0.000
Time: 55.810 ms
- planning: 0.421 ms
- execution: 55.389 ms
- I/O read: 0.000 ms
- I/O write: 0.000 ms
Shared buffers:
- hits: 4931 (~38.50 MiB) from the buffer pool
- reads: 0 from the OS file cache, including disk I/O
- dirtied: 0
- writes: 0
Migration output
main: == 20220602111923 PrepareIndexForOpenIssuesCount: migrating ===================
main: -- index_exists?(:issues, [:project_id, :confidential], {:where=>"state_id = 1", :name=>"idx_open_issues_on_project_id_and_confidential", :algorithm=>:concurrently})
main: -> 0.0069s
main: -- add_index_options(:issues, [:project_id, :confidential], {:where=>"state_id = 1", :name=>"idx_open_issues_on_project_id_and_confidential", :algorithm=>:concurrently})
main: -> 0.0001s
main: == 20220602111923 PrepareIndexForOpenIssuesCount: migrated (0.0177s) ==========
ci: == 20220602111923 PrepareIndexForOpenIssuesCount: migrating ===================
ci: -- index_exists?(:issues, [:project_id, :confidential], {:where=>"state_id = 1", :name=>"idx_open_issues_on_project_id_and_confidential", :algorithm=>:concurrently})
ci: -> 0.0079s
ci: -- add_index_options(:issues, [:project_id, :confidential], {:where=>"state_id = 1", :name=>"idx_open_issues_on_project_id_and_confidential", :algorithm=>:concurrently})
ci: -> 0.0001s
ci: == 20220602111923 PrepareIndexForOpenIssuesCount: migrated (0.0118s) ==========
main: == 20220602111923 PrepareIndexForOpenIssuesCount: reverting ===================
main: == 20220602111923 PrepareIndexForOpenIssuesCount: reverted (0.0085s) ==========
ci: == 20220602111923 PrepareIndexForOpenIssuesCount: reverting ===================
ci: == 20220602111923 PrepareIndexForOpenIssuesCount: reverted (0.0085s) ==========
Related to #363603 (closed)
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Heinrich Lee Yu