Remove the unused join_field from the main Advanced Search index

Summary

The main Advanced Search index mapping declares a join_field (type join) with relation project: [issue, merge_request, milestone, note, blob, wiki_blob, commit], but no search reads it anymore. Project documents have also moved to their own index, so the join has no parent documents in the main index. This removes the dead read path, stops writing the field, and strips it from stored documents, so the mapping can drop it at the next main-index reindex.

Scope: main index only. This issue removes join_field from the main combined Advanced Search index (gitlab-production, mapping in ee/lib/elastic/latest/config.rb). It is the only index mapping that declares join_field, so the separate indices (work items, merge requests, notes, projects, commits, wikis, users, etc.) are not changed and don't need a reindex. This issue covers the code changes and stripping the field from stored documents, none of which needs a reindex. Dropping the field from the mapping happens in the main-index reindex in #600600, which this issue blocks.

Current state

  • Mapping: ee/lib/elastic/latest/config.rb.
  • Dead read path: the only has_parent query is in Elastic::Latest::ApplicationClassProxy#project_ids_filter. It's only reached through #authorization_filter, which has no callers since the git class proxy moved to shared filters in eaf8adc0 ("Remove search_query_authorization_refactor feature flag").
  • Active searches all skip the join:
    • Search::Elastic::QueryBuilder#base_options sets no_join_project: true for every query-builder search, including milestones.
    • Commits are in their own index (Elastic::Latest::CommitConfig.index_name); commit search sets no_join_project: true.
    • Blob search sets it from GitClassProxy#disable_project_joins_for_blob?, which returns true unconditionally.
    • Note search sets no_join_project: true.
  • No has_child queries exist.
  • Project documents are in a separate index (Project.use_separate_indices? is true), routed by n_<root_ancestor_id>.
  • Writers from Rails: ApplicationInstanceProxy#generic_attributes adds join_field whenever es_parent is present. Today that means milestones only: project milestones (parent project_<id>) and, since !257456 (merged), group milestones (parent n_<root_ancestor_id>). Merge requests and notes strip it (super.except('join_field')). Snippets have no es_parent.
  • Deletes don't use it: ElasticDeleteProjectWorker#remove_children_documents deletes by project_id / rid / commit.rid / target_project_id terms with project routing.
  • gitlab-elasticsearch-indexer (Go, separate repo) writes join_field on every blob document (internal/mode/advanced/indexer/indexer.go). Routing doesn't depend on it: the indexer sets project_<id> routing separately (routingFor in internal/mode/advanced/elastic/client.go).

Tasks

  • Remove the dead join path in ApplicationClassProxy: #authorization_filter, #should_use_project_ids_filter?, #project_ids_filter, #project_ids_query and #pick_projects_by_membership have no callers outside that chain. Then remove the no_join_project plumbing that only existed to switch the join off, and update the comment in ee/lib/gitlab/elastic/search_results.rb that points to project_ids_query.
    • !259063 (merged). Removes the whole unused chain, plus helpers only it used. no_join_project stays, because it also controls the doc type filter that snippet search needs.
  • Stop writing join_field from Rails: drop it from ApplicationInstanceProxy#generic_attributes, then remove the except('join_field') overrides in the merge request and note proxies.
  • Stop the Go indexer writing join_field for blobs, and update its test mapping (defaultIndexMapping in internal/mode/advanced/elastic/index.go).
  • Strip join_field from stored documents with a MigrationRemoveFieldsHelper migration (update_by_query with ctx._source.remove), after both writers have stopped. Local testing showed this works while the join mapping still exists and keeps routing (see the 2026-10-01 comment on #600600).
    • !261846. A standalone migration rather than MigrationRemoveFieldsHelper, because the helper handles one document type and join_field is on both blob and milestone documents. Merge after GITLAB_ELASTICSEARCH_INDEXER_VERSION is bumped to 5.14.18 (gitlab-elasticsearch-indexer!960 (closed)), so blobs don't get the field back.
  • Confirm the exists count for join_field is 0 on staging and production, so #600600 can drop it from the mapping.

Notes

  • Existing documents keep the field until they're rewritten. That's harmless while the mapping still has it, but they must be stripped before #600600 drops it from the mapping: the mapping is dynamic: 'strict' and the reindex copies _source as-is, so any document that still has join_field fails its slice and the reindex aborts on the document count check.
  • Came up in #428589 (group milestones in Advanced Search).
Edited by Ravi Kumar