Add sortable title sub-field and title_asc/title_desc for work items
What does this MR do and why?
Adds order_by=title (title_asc / title_desc) to work item search on both the basic (PostgreSQL) and advanced (Elasticsearch) paths.
title is a text field and Elasticsearch cannot sort on text, so the key was excluded with a TODO. This adds a sortable title.keyword sub-field, the mapping and backfill migrations that populate it, and a gate keeping the new keys unavailable until the backfill finishes — a sort on an unpopulated field returns HTTP 200 in tie-breaker order, i.e. silently wrong.
~database: the basic path addsORDER BY lower(title), id DESCand there is no index onlower(title). Cost at large-namespace row counts is unmeasured.- Both paths tie-break on
id DESC, so paging over duplicate titles is stable.
Backfill cost on GitLab.com. The backfill reindexes the whole work_items index. At 38,120,381 primary documents with batch_size 10,000 and throttle_delay 1 minute, the styleguide formula gives 3,812 minutes — expect 2–3 days, during which order_by=title falls back to created_at_desc.
References
- Specification:
specs/advanced-search-filters-and-sorting/SPEC.mdin gitlab-org/search-team/artifacts, merged as6af57ce01254 - #565410
- Database review: the generated basic-search SQL is in $6054339; GDK-scale
EXPLAIN (ANALYZE, BUFFERS)plans are in !254391 (comment 3812030537) and a production-clone plan for theissuespath is in !254391 (comment 3822626022). The MR's only change to that query is theORDER BYclause.
@johnmason for feedback, or reply #human to escalate to John.
Mechanism and citations (for agents / deep readers)
- Mapping —
ee/lib/search/elastic/types/work_item.rb:title.fields.keyword,normalizer: lower_case_normalizer,ignore_above: 1024.Issuable::TITLE_LENGTH_MAXcaps titles at 255, so nothing reachable is dropped; an over-limit title is still indexed and sorts into the missing tail. Elasticsearch's normalizer and PostgreSQL'slower(title)disagree on non-ASCII titles; measured examples are in the code comment above the mapping in that file. - Migrations —
AddTitleKeywordToWorkItemsoverridescompleted?becauseMigrationUpdateMappingsHelper#completed?compares only top-level keys andtitlealready exists.BackfillTitleKeywordInWorkItemscarriesretry_on_failure, so a persistently failing run reachesmigration.failinstead of parking every later migration behind it. Itsbatch_sizeandthrottle_delaymatch four siblingwork_itemsreindex migrations. - Document count — 38,120,381 is
elasticsearch_indices_docs_primaryforgitlab-production-work_items-20251016-1225-reindex-2000233-0in the gprd Mimir datasource, latest scrape 2026-06-14. Primary shards only, so it counts each document once;elasticsearch_indices_docs_totalreads 76,240,762 across primaries and one replica. - Runtime figure is a ceiling —
process_batch_with_scroll!loops untiltotal_processedreachesQUEUE_THRESHOLD(50,000), so one run can enqueue more thanbatch_size. - Gate —
ee/lib/search/elastic/sorts/base.rb:sort_available?lets the advanced finders route to PostgreSQL, which is correct in every migration state;build_sortsubstitutesFALLBACK_SORT_KEY = :created_at_descand logs for callers that cannot re-route. The fallback must be a key every subclass defines, becausePaginationreadsquery_hash[:sort]unconditionally. The pre-backfill behaviour is documented indoc/api/search.md. - Basic search —
lib/gitlab/search_results.rbcallsorder_title_asc.with_order_id_desc/order_title_desc.with_order_id_desc, mirroringIssuable.sort_by_attribute. The sort is a bounded top-N above the already-filtered join, capped bySearchService::MAX_PER_PAGE.
What I did not verify
- No postgres.ai access: the query plans are GDK-scale only. The document count is a production metric, but its latest scrape is 2026-06-14, so the index has grown since.
- CI's pinned Elasticsearch versions (7.17 / 8.19 / 9.5) were not exercised locally; local runs used 9.4.4.
- The GraphQL resolver path was not exercised end to end.