Fetching labels from project and ancestor groups during project transfer may timeout with cold cache
During transfer of project gitlabdemo-cloud-app from group gitlab-com/business-technology/engineering/infrastructure/managed-services/sales-cs-demo-systems/management-apps to a different one, while trying to transfer labels we've hit a statement timeout while trying to fetch issues labels for project and its ancestors, even though there are only 13 issues in that project, similar the second statement timeout was hit on fetching labels for merge requests with only 10 MRs.
Cold cache sql
explain (analyze, buffers) SELECT label_links.id FROM labels INNER JOIN label_links ON label_links.target_type = 'Issue' AND label_links.label_id = labels.id INNER JOIN issues ON issues.id = label_links.target_id WHERE issues.project_id = 26350347 AND labels.group_id IN (SELECT namespaces.id FROM namespaces WHERE namespaces.id IN (6543, 3587891, 12398626, 12398637, 12399085, 12515747, 11923824)) ORDER BY labels.title ASC;
QUERY PLAN
--------------------------------------------------------------------------------------------
Sort (cost=355.19..355.20 rows=1 width=92) (actual time=24934.761..24934.766 rows=0 loops=1)
Sort Key: labels.title
Sort Method: quicksort Memory: 25kB
Buffers: shared hit=189872 read=10353 dirtied=201
I/O Timings: read=7682.797
-> Hash Join (cost=15.31..355.18 rows=1 width=92) (actual time=24934.732..24934.736 rows=0 loops=1)
Hash Cond: (label_links.target_id = issues.id)
Buffers: shared hit=189869 read=10353 dirtied=201
I/O Timings: read=7682.797
-> Nested Loop (cost=1.56..341.30 rows=50 width=96) (actual time=0.065..24885.589 rows=203033 loops=1)
Buffers: shared hit=189848 read=10353 dirtied=201
I/O Timings: read=7682.797
-> Nested Loop (cost=1.00..234.85 rows=9 width=92) (actual time=0.035..166.098 rows=1602 loops=1)
Buffers: shared hit=1170 read=80
I/O Timings: read=50.061
-> Index Only Scan using namespaces_pkey on namespaces (cost=0.43..15.17 rows=7 width=4) (actual time=0.019..11.097 rows=7 loops=1)
Index Cond: (id = ANY ('{6543,3587891,12398626,12398637,12399085,12515747,11923824}'::integer[]))
Heap Fetches: 1
Buffers: shared hit=26 read=2
I/O Timings: read=3.497
-> Index Scan using index_labels_on_group_id on labels (cost=0.56..31.15 rows=23 width=92) (actual time=0.716..22.040 rows=229 loops=7)
Index Cond: (group_id = namespaces.id)
Buffers: shared hit=1144 read=78
I/O Timings: read=46.564
-> Index Scan using index_label_links_on_label_id_and_target_type on label_links (cost=0.57..8.12 rows=371 width=8) (actual time=0.457..15.379 rows=127 loops=1602)
Index Cond: ((label_id = labels.id) AND ((target_type)::text = 'Issue'::text))
Buffers: shared hit=188678 read=10273 dirtied=201
I/O Timings: read=7632.736
-> Hash (cost=11.58..11.58 rows=173 width=4) (actual time=0.050..0.051 rows=13 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 9kB
Buffers: shared hit=18
-> Index Only Scan using idx_issues_on_project_id_and_updated_at_and_id_and_state_id on issues (cost=0.56..11.58 rows=173 width=4) (actual time=0.021..0.036 rows=13 loops=1)
Index Cond: (project_id = 26350347)
Heap Fetches: 1
Buffers: shared hit=18
Planning Time: 5.776 ms
Execution Time: 24934.881 ms
(37 rows)
Edited by Alexandru Croitor