Add BBM to nullify personal_namespace_id on group wiki events

What does this MR do and why?

Adds a batched background migration (BBM) to clean up events rows where both group_id and personal_namespace_id are set for target_type = 'WikiPage::Meta'.

Background

These rows are historical leftovers from a combination of two changes:

  1. Bug window (GitLab 17.5-17.6, Oct 7 - Nov 6 2024): Group wiki events were incorrectly stored with personal_namespace_id instead of group_id because WikiPage::Meta#resource_parent returned nil for group wikis (it aliased project, which is nil for group wikis).

  2. December 2024 backfill (BackfillGroupWikiActivityEvents): Set group_id on those broken rows but did not clear personal_namespace_id, leaving both sharding key columns populated simultaneously.

No current code path produces such rows - this is purely historical data cleanup.

Solution

This BBM iterates over the events table and nullifies personal_namespace_id on rows where:

  • target_type = 'WikiPage::Meta'
  • group_id IS NOT NULL
  • personal_namespace_id IS NOT NULL

This ensures each event has exactly one sharding key set, which is required for the NOT NULL constraint tightening tracked in #598786.

The migration uses scope_to with the target_type, group_id, and personal_namespace_id conditions, so the batching query itself is filtered instead of paging through the whole events table by primary key. This scope is covered by two existing partial indexes, index_events_on_group_id_and_id and index_events_on_personal_namespace_id, and the planner combines them with a BitmapAnd. The Database/AvoidScopeTo cop is disabled on the scope_to block, with both indexes named in the disable comment as the docs require.

Queries and query plans

These are the queries the BBM issues, with a batch size of 10,000 and a sub-batch size of 500.

Batching query, upper bound lookup:

SELECT "events"."id" FROM "events" WHERE "events"."id" >= 1 AND "events"."target_type" = 'WikiPage::Meta' AND "events"."group_id" IS NOT NULL AND "events"."personal_namespace_id" IS NOT NULL ORDER BY "events"."id" ASC LIMIT 1 OFFSET 10000

Batching query, MIN/MAX pick over the batch:

SELECT MIN(events.id), MAX(events.id) FROM "events" WHERE "events"."id" >= 1 AND "events"."target_type" = 'WikiPage::Meta' AND "events"."group_id" IS NOT NULL AND "events"."personal_namespace_id" IS NOT NULL ORDER BY "events"."id" ASC LIMIT 10000

Sub-batch bound lookup:

SELECT "events"."id" FROM "events" WHERE "events"."id" BETWEEN 1 AND 10000 AND "events"."target_type" = 'WikiPage::Meta' AND "events"."group_id" IS NOT NULL AND "events"."personal_namespace_id" IS NOT NULL ORDER BY "events"."id" ASC LIMIT 1 OFFSET 500

Sub-batch UPDATE:

UPDATE "events" SET "personal_namespace_id" = NULL WHERE "events"."id" BETWEEN 1 AND 500 AND "events"."target_type" = 'WikiPage::Meta' AND "events"."group_id" IS NOT NULL AND "events"."personal_namespace_id" IS NOT NULL

The 1, 10000, and 500 bounds above are illustrative first-batch values; real runs substitute the current batch bounds as the migration progresses.

scope_to defines filter_batch, and PrimaryKeyBatchingStrategy#next_batch calls it, so the three conditions apply to the batching query as well as to each sub-batch. That is why perform no longer repeats them.

Query plans

Plans for the equivalent raw statements were run against GitLab.com data as part of the schema change exception request, which is confidential, so the detail lives there: https://gitlab.com/gitlab-org/database-team/team-tasks/-/issues/679. The outcome is that the filter resolves to a BitmapAnd over the two existing partial indexes index_events_on_group_id_and_id and index_events_on_personal_namespace_id, and roughly 14,000 rows match in total. That was enough to conclude no new index is needed for this migration.

One tradeoff to call out explicitly, since it follows from the public index definitions alone: neither index contains target_type, so target_type = 'WikiPage::Meta' cannot be an index condition and is applied as a residual filter after the bitmap heap recheck. The docs describe the ideal signal for scope_to as an index-only scan with no additional filters, and that is genuinely not met here. What makes it acceptable is the row count: with roughly 14,000 matching rows in total and a batch size of 10,000, the whole migration is only a couple of batches, so the batching query runs a handful of times rather than continuously.

What is still outstanding is a review of the plans for the queries above, the ones the BBM framework itself generates, as opposed to the hand written equivalents already measured. That is the reason the exact SQL is listed here. @l.rosa has been asked for that review.

References

Screenshots or screen recordings

N/A - database migration only.

Before After
N/A N/A

How to set up and validate locally

  1. In a Rails console, find events with both sharding keys set:
    Event.where(target_type: 'WikiPage::Meta').where.not(group_id: nil).where.not(personal_namespace_id: nil).count
  2. Run the migration and verify the count drops to 0.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Chen Zhang

Merge request reports

Loading
Loading