Copy p_sent_notifications when moving issues
What does this MR do and why?
Use p_sent_notifications table when moving issue records as part of the effort to drop the original sent_notifications table
Notes for reviewer
- Traversing a partitioned table in batches is not idea here, but I don't think we'll ever have to deal with huge amount of records associated to a single issue, specially because we are only copying records associated to service desk issues.
- Now we have 2 partitions in .com. But in the future we will have up to 12 partitions at any given time
- Dropping the unique constraint was discussed in #577844 (comment 2838135886) so this MR also implements the mechanism to handle an unlikely collision
Migration output
UP
main: == [advisory_lock_connection] object_id: 130880, pg_backend_pid: 15732
main: == 20251126165557 DropPSentNotificationsReplyKeyUniqueIndex: migrating ========
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- index_name_exists?(:p_sent_notifications, "index_p_sent_notifications_on_reply_key_partition_unique")
main: -> 0.0013s
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- remove_index(:p_sent_notifications, {:name=>"index_p_sent_notifications_on_reply_key_partition_unique"})
main: -> 0.0010s
main: == 20251126165557 DropPSentNotificationsReplyKeyUniqueIndex: migrated (0.1127s)
main: == [advisory_lock_connection] object_id: 130880, pg_backend_pid: 15732
ci: == [advisory_lock_connection] object_id: 130880, pg_backend_pid: 15734
ci: == 20251126165557 DropPSentNotificationsReplyKeyUniqueIndex: migrating ========
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- index_name_exists?(:p_sent_notifications, "index_p_sent_notifications_on_reply_key_partition_unique")
ci: -> 0.0020s
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- remove_index(:p_sent_notifications, {:name=>"index_p_sent_notifications_on_reply_key_partition_unique"})
ci: -> 0.0016s
ci: == 20251126165557 DropPSentNotificationsReplyKeyUniqueIndex: migrated (0.0219s)
ci: == [advisory_lock_connection] object_id: 130880, pg_backend_pid: 15734
DOWN
main: == [advisory_lock_connection] object_id: 130400, pg_backend_pid: 16141
main: == 20251126165557 DropPSentNotificationsReplyKeyUniqueIndex: reverting ========
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- index_name_exists?(:p_sent_notifications, "index_p_sent_notifications_on_reply_key_partition_unique")
main: -> 0.0010s
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- index_exists?("gitlab_partitions_dynamic.p_sent_notifications_1", [:reply_key, :partition], {:unique=>true, :name=>"index_a6de33f55a", :algorithm=>:concurrently})
main: -> 0.0018s
main: -- execute("SET statement_timeout TO 0")
main: -> 0.0005s
main: -- add_index("gitlab_partitions_dynamic.p_sent_notifications_1", [:reply_key, :partition], {:unique=>true, :name=>"index_a6de33f55a", :algorithm=>:concurrently})
main: -> 0.0040s
main: -- execute("RESET statement_timeout")
main: -> 0.0003s
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- index_exists?("gitlab_partitions_dynamic.p_sent_notifications_2", [:reply_key, :partition], {:unique=>true, :name=>"index_cf8d093394", :algorithm=>:concurrently})
main: -> 0.0015s
main: -- add_index("gitlab_partitions_dynamic.p_sent_notifications_2", [:reply_key, :partition], {:unique=>true, :name=>"index_cf8d093394", :algorithm=>:concurrently})
main: -> 0.0022s
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- add_index(:p_sent_notifications, [:reply_key, :partition], {:unique=>true, :name=>"index_p_sent_notifications_on_reply_key_partition_unique"})
main: -> 0.0015s
main: == 20251126165557 DropPSentNotificationsReplyKeyUniqueIndex: reverted (0.1258s)
main: == [advisory_lock_connection] object_id: 130400, pg_backend_pid: 16141
ci: == [advisory_lock_connection] object_id: 130400, pg_backend_pid: 16163
ci: == 20251126165557 DropPSentNotificationsReplyKeyUniqueIndex: reverting ========
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- index_name_exists?(:p_sent_notifications, "index_p_sent_notifications_on_reply_key_partition_unique")
ci: -> 0.0013s
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- index_exists?("gitlab_partitions_dynamic.p_sent_notifications_1", [:reply_key, :partition], {:unique=>true, :name=>"index_a6de33f55a", :algorithm=>:concurrently})
ci: -> 0.0026s
ci: -- execute("SET statement_timeout TO 0")
ci: -> 0.0004s
ci: -- add_index("gitlab_partitions_dynamic.p_sent_notifications_1", [:reply_key, :partition], {:unique=>true, :name=>"index_a6de33f55a", :algorithm=>:concurrently})
ci: -> 0.0066s
ci: -- execute("RESET statement_timeout")
ci: -> 0.0004s
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- index_exists?("gitlab_partitions_dynamic.p_sent_notifications_2", [:reply_key, :partition], {:unique=>true, :name=>"index_cf8d093394", :algorithm=>:concurrently})
ci: -> 0.0016s
ci: -- add_index("gitlab_partitions_dynamic.p_sent_notifications_2", [:reply_key, :partition], {:unique=>true, :name=>"index_cf8d093394", :algorithm=>:concurrently})
ci: -> 0.0020s
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- add_index(:p_sent_notifications, [:reply_key, :partition], {:unique=>true, :name=>"index_p_sent_notifications_on_reply_key_partition_unique"})
ci: -> 0.0013s
ci: == 20251126165557 DropPSentNotificationsReplyKeyUniqueIndex: reverted (0.0436s)
ci: == [advisory_lock_connection] object_id: 130400, pg_backend_pid: 16163
Query plans
First id on partitioned table
https://console.postgres.ai/gitlab/gitlab-production-main/sessions/45896/commands/140419
SELECT id
FROM p_sent_notifications
WHERE noteable_type = 'Issue'
AND noteable_id = 130452526
ORDER BY id
LIMIT 1
Fetch partitioned batch upper bound
https://console.postgres.ai/gitlab/gitlab-production-main/sessions/45896/commands/140420
SELECT id
FROM p_sent_notifications
WHERE noteable_type = 'Issue'
AND noteable_id = 130452526
ORDER BY id offset 100 limit 1
Fetch all records from batch
https://console.postgres.ai/gitlab/gitlab-production-main/sessions/45896/commands/140422
SELECT * FROM "p_sent_notifications"
WHERE "p_sent_notifications"."noteable_id" = 130452526
AND "p_sent_notifications"."noteable_type" = 'Issue'
AND "p_sent_notifications"."id" >= 2724911326
AND "p_sent_notifications"."id" < 2725663111
Delete all records from batch
https://console.postgres.ai/gitlab/gitlab-production-main/sessions/45896/commands/140423
DELETE FROM "p_sent_notifications"
WHERE "p_sent_notifications"."noteable_id" = 130452526
AND "p_sent_notifications"."noteable_type" = 'Issue'
AND "p_sent_notifications"."id" >= 2724911326
AND "p_sent_notifications"."id" < 2725663111
Related to #577844 (closed)
Edited by Mario Celi