Advanced search - add new fields to work item index

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

Description

We currently need to support filtering by the following fields (please refer to GLQL fields docs for more info):

• iteration
• iteration cadence
• custom status
• custom field
• subscribed
• emoji reaction
• title

So these fields need to be update to the ES work items index. It will require some investigation to decide how to store these field in ES and what data type they should be (it depends on how this data is also stored in postgres).

❗ Note: I'd prefer doing this in one go since it add a check whether a migration has run to the code and we already have a bunch of those which gets a bit more difficult to keep a track of. However, this issue can be broken down into smaller steps to add mappings and migrate each field.

Implementation

Follow the documentation on how to add new fields to the work_items index, populate the new fields and keep the records in sync (https://docs.gitlab.com/development/advanced_search/#adding-a-field).

Part 1: Adding new field and keep updated

• Add new fields to the index mapping in the base_mappings method in Search::Elastic::Types::WorkItem
• Add an advanced search migration to add the new field using the MigrationUpdateMappingsHelper
• In Search::Elastic::References::WorkItem
  • populate the new field in the build_indexed_json method, add a gate to ensure the migration created above has completed
  • bump the SCHEMA_VERSION to a new value (YYYYWW, year and week number)
• Check whether this preload_indexing_data N+1 spec should be updated for iteratiom, cadence and parent link since we are now indexing work items relations:
  • the specs for indexing are ee/spec/services/elastic/process_bookkeeping_service_spec.rb
• Sample MR

Part 2: Backfilling data

• Add an advanced search migration to populate the new field using the MigrationBackfillHelper
  • use MigrationReindexBasedOnSchemaVersion instead of MigrationBackfillHelper since these fields are nullable
• sample MR