Copy p_sent_notifications when moving issues

What does this MR do and why?

Use p_sent_notifications table when moving issue records as part of the effort to drop the original sent_notifications table

Notes for reviewer

  • Traversing a partitioned table in batches is not idea here, but I don't think we'll ever have to deal with huge amount of records associated to a single issue, specially because we are only copying records associated to service desk issues.
  • Now we have 2 partitions in .com. But in the future we will have up to 12 partitions at any given time
  • Dropping the unique constraint was discussed in #577844 (comment 2838135886) so this MR also implements the mechanism to handle an unlikely collision

Migration output

UP
main: == [advisory_lock_connection] object_id: 130880, pg_backend_pid: 15732
main: == 20251126165557 DropPSentNotificationsReplyKeyUniqueIndex: migrating ========
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- index_name_exists?(:p_sent_notifications, "index_p_sent_notifications_on_reply_key_partition_unique")
main:    -> 0.0013s
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- remove_index(:p_sent_notifications, {:name=>"index_p_sent_notifications_on_reply_key_partition_unique"})
main:    -> 0.0010s
main: == 20251126165557 DropPSentNotificationsReplyKeyUniqueIndex: migrated (0.1127s) 

main: == [advisory_lock_connection] object_id: 130880, pg_backend_pid: 15732
ci: == [advisory_lock_connection] object_id: 130880, pg_backend_pid: 15734
ci: == 20251126165557 DropPSentNotificationsReplyKeyUniqueIndex: migrating ========
ci: -- transaction_open?(nil)
ci:    -> 0.0000s
ci: -- index_name_exists?(:p_sent_notifications, "index_p_sent_notifications_on_reply_key_partition_unique")
ci:    -> 0.0020s
ci: -- transaction_open?(nil)
ci:    -> 0.0000s
ci: -- remove_index(:p_sent_notifications, {:name=>"index_p_sent_notifications_on_reply_key_partition_unique"})
ci:    -> 0.0016s
ci: == 20251126165557 DropPSentNotificationsReplyKeyUniqueIndex: migrated (0.0219s) 

ci: == [advisory_lock_connection] object_id: 130880, pg_backend_pid: 15734
DOWN
main: == [advisory_lock_connection] object_id: 130400, pg_backend_pid: 16141
main: == 20251126165557 DropPSentNotificationsReplyKeyUniqueIndex: reverting ========
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- index_name_exists?(:p_sent_notifications, "index_p_sent_notifications_on_reply_key_partition_unique")
main:    -> 0.0010s
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- index_exists?("gitlab_partitions_dynamic.p_sent_notifications_1", [:reply_key, :partition], {:unique=>true, :name=>"index_a6de33f55a", :algorithm=>:concurrently})
main:    -> 0.0018s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0005s
main: -- add_index("gitlab_partitions_dynamic.p_sent_notifications_1", [:reply_key, :partition], {:unique=>true, :name=>"index_a6de33f55a", :algorithm=>:concurrently})
main:    -> 0.0040s
main: -- execute("RESET statement_timeout")
main:    -> 0.0003s
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- index_exists?("gitlab_partitions_dynamic.p_sent_notifications_2", [:reply_key, :partition], {:unique=>true, :name=>"index_cf8d093394", :algorithm=>:concurrently})
main:    -> 0.0015s
main: -- add_index("gitlab_partitions_dynamic.p_sent_notifications_2", [:reply_key, :partition], {:unique=>true, :name=>"index_cf8d093394", :algorithm=>:concurrently})
main:    -> 0.0022s
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- add_index(:p_sent_notifications, [:reply_key, :partition], {:unique=>true, :name=>"index_p_sent_notifications_on_reply_key_partition_unique"})
main:    -> 0.0015s
main: == 20251126165557 DropPSentNotificationsReplyKeyUniqueIndex: reverted (0.1258s) 

main: == [advisory_lock_connection] object_id: 130400, pg_backend_pid: 16141
ci: == [advisory_lock_connection] object_id: 130400, pg_backend_pid: 16163
ci: == 20251126165557 DropPSentNotificationsReplyKeyUniqueIndex: reverting ========
ci: -- transaction_open?(nil)
ci:    -> 0.0000s
ci: -- index_name_exists?(:p_sent_notifications, "index_p_sent_notifications_on_reply_key_partition_unique")
ci:    -> 0.0013s
ci: -- transaction_open?(nil)
ci:    -> 0.0000s
ci: -- index_exists?("gitlab_partitions_dynamic.p_sent_notifications_1", [:reply_key, :partition], {:unique=>true, :name=>"index_a6de33f55a", :algorithm=>:concurrently})
ci:    -> 0.0026s
ci: -- execute("SET statement_timeout TO 0")
ci:    -> 0.0004s
ci: -- add_index("gitlab_partitions_dynamic.p_sent_notifications_1", [:reply_key, :partition], {:unique=>true, :name=>"index_a6de33f55a", :algorithm=>:concurrently})
ci:    -> 0.0066s
ci: -- execute("RESET statement_timeout")
ci:    -> 0.0004s
ci: -- transaction_open?(nil)
ci:    -> 0.0000s
ci: -- index_exists?("gitlab_partitions_dynamic.p_sent_notifications_2", [:reply_key, :partition], {:unique=>true, :name=>"index_cf8d093394", :algorithm=>:concurrently})
ci:    -> 0.0016s
ci: -- add_index("gitlab_partitions_dynamic.p_sent_notifications_2", [:reply_key, :partition], {:unique=>true, :name=>"index_cf8d093394", :algorithm=>:concurrently})
ci:    -> 0.0020s
ci: -- transaction_open?(nil)
ci:    -> 0.0000s
ci: -- add_index(:p_sent_notifications, [:reply_key, :partition], {:unique=>true, :name=>"index_p_sent_notifications_on_reply_key_partition_unique"})
ci:    -> 0.0013s
ci: == 20251126165557 DropPSentNotificationsReplyKeyUniqueIndex: reverted (0.0436s) 

ci: == [advisory_lock_connection] object_id: 130400, pg_backend_pid: 16163

Query plans

First id on partitioned table

https://console.postgres.ai/gitlab/gitlab-production-main/sessions/45896/commands/140419

SELECT id
FROM   p_sent_notifications
WHERE  noteable_type = 'Issue'
       AND noteable_id = 130452526
ORDER  BY id
LIMIT  1
Fetch partitioned batch upper bound

https://console.postgres.ai/gitlab/gitlab-production-main/sessions/45896/commands/140420

SELECT   id
FROM     p_sent_notifications
WHERE    noteable_type = 'Issue'
AND      noteable_id = 130452526
ORDER BY id offset 100 limit 1
Fetch all records from batch

https://console.postgres.ai/gitlab/gitlab-production-main/sessions/45896/commands/140422

SELECT * FROM "p_sent_notifications"
WHERE  "p_sent_notifications"."noteable_id" = 130452526
       AND "p_sent_notifications"."noteable_type" = 'Issue'
       AND "p_sent_notifications"."id" >= 2724911326
       AND "p_sent_notifications"."id" < 2725663111
Delete all records from batch

https://console.postgres.ai/gitlab/gitlab-production-main/sessions/45896/commands/140423

DELETE FROM "p_sent_notifications"
WHERE  "p_sent_notifications"."noteable_id" = 130452526
       AND "p_sent_notifications"."noteable_type" = 'Issue'
       AND "p_sent_notifications"."id" >= 2724911326
       AND "p_sent_notifications"."id" < 2725663111

Related to #577844 (closed)

Edited by Mario Celi

Merge request reports

Loading