Backfill project access data on snippet documents

What does this MR do and why?

Step 2 of 3 for #35760 (Advanced Search never returns public/internal project snippets to non-members).

Builds on !259626 (merged), which has merged.

Step 1 adds the project_visibility_level mapping. Once its mapping migration (AddProjectVisibilityLevelToSnippets) has finished, it writes project_visibility_level and snippets_access_level on project snippet documents. It also bumps the snippet SCHEMA_VERSION to 26_02. Documents keep 26_01 until that migration finishes, so no document claims 26_02 without the fields.

Project snippet documents already in production were indexed before step 1, so they have no project fields. They only get them when they are reindexed (a snippet edit, or a change to project visibility or snippets access level). Without a backfill, the step 3 filter could not match most public and internal project snippets.

This MR adds that backfill.

Advanced Search migration

BackfillProjectAccessOnSnippets uses Search::Elastic::MigrationReindexBasedOnSchemaVersion with:

  • DOCUMENT_TYPE = Snippet
  • NEW_SCHEMA_VERSION = 26_02
  • batched!, batch_size 10_000, throttle_delay 1.minute
  • retry_on_failure

The helper filters on type: snippet, so only snippet documents in the main index are reindexed. Personal snippets are reindexed too. They get no new fields, but they move to 26_02, so the remaining-document count reaches zero.

Ordering. Migrations run in version order, so this one runs after the step 1 mapping migration (20261005110000, then 20261013103632).

Multi-version compatibility. Before the mapping migration finishes, no document reaches 26_02. The backfill therefore never completes early.

Runtime. The planner estimates about 775,000 rows in snippets on GitLab.com (postgres.ai plan, no execution). At 10,000 documents per batch and a 1-minute throttle, that is about 78 batches, or roughly 80 minutes. It takes longer if the bookkeeping queue is above the helper's 50,000 threshold, because the helper skips a run when the queue is full.

Reversibility. A reindex only rewrites documents from the database. Nothing is deleted. It is safe to rerun.

Next step. Step 3 adds the public/internal branch in filter_project_snippets, gated on this backfill finishing.

Tests

  • The shared migration reindex based on schema_version examples, run with project and personal snippets.
  • A spec that strips project_visibility_level and snippets_access_level from a project snippet document and resets its schema version. It then runs the migration and checks that the fields come back. This spec fails if the migration does nothing.

References

Screenshots or screen recordings

Not applicable. This MR has no UI changes.

How to set up and validate locally

  1. Enable Advanced Search in GDK and index a project snippet.
  2. Set the project snippet document's schema_version to 23_08 and remove project_visibility_level. For example, use an update-by-query.
  3. Run Elastic::MigrationWorker until Elastic::DataMigrationService.migration_has_finished?(:backfill_project_access_on_snippets) returns true.
  4. Fetch the document snippet_<id> and check that project_visibility_level and snippets_access_level are back.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

🤖 Generated with Claude Code

Edited by Ravi Kumar

Merge request reports

Loading
Loading