Backfill project access data on snippet documents
What does this MR do and why?
Step 2 of 3 for #35760 (Advanced Search never returns public/internal project snippets to non-members).
Builds on !259626 (merged), which has merged.
Step 1 adds the project_visibility_level mapping. Once its mapping migration (AddProjectVisibilityLevelToSnippets) has finished, it writes project_visibility_level and snippets_access_level on project snippet documents. It also bumps the snippet SCHEMA_VERSION to 26_02. Documents keep 26_01 until that migration finishes, so no document claims 26_02 without the fields.
Project snippet documents already in production were indexed before step 1, so they have no project fields. They only get them when they are reindexed (a snippet edit, or a change to project visibility or snippets access level). Without a backfill, the step 3 filter could not match most public and internal project snippets.
This MR adds that backfill.
Advanced Search migration
BackfillProjectAccessOnSnippets uses Search::Elastic::MigrationReindexBasedOnSchemaVersion with:
DOCUMENT_TYPE = SnippetNEW_SCHEMA_VERSION = 26_02batched!,batch_size 10_000,throttle_delay 1.minuteretry_on_failure
The helper filters on type: snippet, so only snippet documents in the main index are reindexed. Personal snippets are reindexed too. They get no new fields, but they move to 26_02, so the remaining-document count reaches zero.
Ordering. Migrations run in version order, so this one runs after the step 1 mapping migration (20261005110000, then 20261013103632).
Multi-version compatibility. Before the mapping migration finishes, no document reaches 26_02. The backfill therefore never completes early.
Runtime. The planner estimates about 775,000 rows in snippets on GitLab.com (postgres.ai plan, no execution). At 10,000 documents per batch and a 1-minute throttle, that is about 78 batches, or roughly 80 minutes. It takes longer if the bookkeeping queue is above the helper's 50,000 threshold, because the helper skips a run when the queue is full.
Reversibility. A reindex only rewrites documents from the database. Nothing is deleted. It is safe to rerun.
Next step. Step 3 adds the public/internal branch in filter_project_snippets, gated on this backfill finishing.
Tests
- The shared
migration reindex based on schema_versionexamples, run with project and personal snippets. - A spec that strips
project_visibility_levelandsnippets_access_levelfrom a project snippet document and resets its schema version. It then runs the migration and checks that the fields come back. This spec fails if the migration does nothing.
References
- Work item: #35760
- Step 1: !259626 (merged)
Screenshots or screen recordings
Not applicable. This MR has no UI changes.
How to set up and validate locally
- Enable Advanced Search in GDK and index a project snippet.
- Set the project snippet document's
schema_versionto23_08and removeproject_visibility_level. For example, use an update-by-query. - Run
Elastic::MigrationWorkeruntilElastic::DataMigrationService.migration_has_finished?(:backfill_project_access_on_snippets)returns true. - Fetch the document
snippet_<id>and check thatproject_visibility_levelandsnippets_access_levelare back.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.