Backfill slack_integrations table in batches
What does this MR do and why?
This MR creates a batched background migration to backfill the sharding keys on the slack_integrations table.
References
Queries
SELECT
"slack_integrations"."id"
FROM
"slack_integrations"
WHERE
"slack_integrations"."id" BETWEEN 23 AND 126
ORDER BY
"slack_integrations"."id" ASC
LIMIT 1
Limit (cost=0.42..0.55 rows=1 width=4) (actual time=0.011..0.011 rows=1 loops=1)
Buffers: shared hit=4
I/O Timings: read=0.000 write=0.000
-> Index Only Scan using slack_integrations_pkey on public.slack_integrations (cost=0.42..8.43 rows=63 width=4) (actual time=0.009..0.009 rows=1 loops=1)
Index Cond: ((slack_integrations.id >= 23) AND (slack_integrations.id <= 126))
Heap Fetches: 0
Buffers: shared hit=4
I/O Timings: read=0.000 write=0.000
Settings: effective_cache_size = '472585MB', jit = 'off', random_page_cost = '1.5', seq_page_cost = '4', work_mem = '100MB'
https://postgres.ai/console/gitlab/gitlab-production-main/sessions/45268/commands/138829
SELECT
"slack_integrations"."id"
FROM
"slack_integrations"
WHERE
"slack_integrations"."id" BETWEEN 23 AND 126
AND "slack_integrations"."id" >= 23
ORDER BY
"slack_integrations"."id" ASC
LIMIT 1 OFFSET 1
Limit (cost=0.55..0.68 rows=1 width=4) (actual time=0.022..0.023 rows=1 loops=1)
Buffers: shared hit=4
I/O Timings: read=0.000 write=0.000
-> Index Only Scan using slack_integrations_pkey on public.slack_integrations (cost=0.42..8.59 rows=63 width=4) (actual time=0.020..0.021 rows=2 loops=1)
Index Cond: ((slack_integrations.id >= 23) AND (slack_integrations.id <= 126) AND (slack_integrations.id >= 23))
Heap Fetches: 0
Buffers: shared hit=4
I/O Timings: read=0.000 write=0.000
Settings: seq_page_cost = '4', work_mem = '100MB', effective_cache_size = '472585MB', jit = 'off', random_page_cost = '1.5'
https://postgres.ai/console/gitlab/gitlab-production-main/sessions/45268/commands/138830
SELECT
"slack_integrations"."integration_id"
FROM
"slack_integrations"
WHERE
"slack_integrations"."id" BETWEEN 23 AND 126
AND "slack_integrations"."id" >= 23
AND "slack_integrations"."id" < 24
AND "slack_integrations"."project_id" IS NULL
AND "slack_integrations"."group_id" IS NULL
AND "slack_integrations"."organization_id" IS NULL
Index Scan using slack_integrations_pkey on public.slack_integrations (cost=0.42..3.45 rows=1 width=4) (actual time=0.004..0.004 rows=0 loops=1)
Index Cond: ((slack_integrations.id >= 23) AND (slack_integrations.id <= 126) AND (slack_integrations.id >= 23) AND (slack_integrations.id < 24))
Filter: ((slack_integrations.project_id IS NULL) AND (slack_integrations.group_id IS NULL) AND (slack_integrations.organization_id IS NULL))
Rows Removed by Filter: 0
Buffers: shared hit=3
I/O Timings: read=0.000 write=0.000
Settings: effective_cache_size = '472585MB', jit = 'off', random_page_cost = '1.5', seq_page_cost = '4', work_mem = '100MB'
https://console.postgres.ai/gitlab/gitlab-production-main/sessions/45268/commands/138834
UPDATE
"slack_integrations"
SET
"updated_at" = '2025-11-06 13:26:57.176953',
"organization_id" = 1
WHERE
"slack_integrations"."id" = 23
ModifyTable on public.slack_integrations (cost=0.42..3.44 rows=0 width=0) (actual time=0.023..0.024 rows=0 loops=1)
Buffers: shared hit=3
I/O Timings: read=0.000 write=0.000
-> Index Scan using slack_integrations_pkey on public.slack_integrations (cost=0.42..3.44 rows=1 width=22) (actual time=0.022..0.022 rows=0 loops=1)
Index Cond: (slack_integrations.id = 23)
Buffers: shared hit=3
I/O Timings: read=0.000 write=0.000
Settings: random_page_cost = '1.5', seq_page_cost = '4', work_mem = '100MB', effective_cache_size = '472585MB', jit = 'off'
https://postgres.ai/console/gitlab/gitlab-production-main/sessions/45268/commands/138842
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Edited by James Nutt