<!--IssueSummary start--> <details> <summary> Everyone can contribute. [Help move this issue forward](https://handbook.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows/#contributor-links) while earning points, leveling up and collecting rewards. </summary> - [Label this issue](https://contributors.gitlab.com/manage-issue?action=label&projectId=278964&issueIid=607015) </details> <!--IssueSummary end--> # Task 1: Adding Group Index to Elasticsearch ## Context Currently, the group dropdown in global search uses the Groups API (`Api.groups()`) which queries the database directly. This causes performance issues and increased database load. We need to index group names in Elasticsearch for faster searches. **Reference MR**: !241326 (sbom_occurrence_refs index) ## Implementation Details ### 1. Add ES Methods to Group Model **File**: `app/models/group.rb` or `ee/app/models/ee/group.rb` ```ruby scope :preload_indexing_data, -> { preload( :route, :parent, :namespace_settings ) } def self.generate_es_parent(group) "root_namespace_#{group.traversal_ids.first}" end def es_parent self.class.generate_es_parent(self) end ``` ### 2. Create Group Reference Class **File**: `ee/lib/search/elastic/references/group.rb` Follow exact pattern from `references/sbom/occurrence_ref.rb`: - Inherit from `Search::Elastic::Reference` - Include `Search::Elastic::Concerns::DatabaseReference` - Define constants: - `DOC_TYPE = 'group'` - `INDEX_NAME = 'groups'` - `SCHEMA_VERSIONS = { 26_XX => nil }.freeze` - `DIRECT_FIELDS` array for fields from groups table - Implement class methods: - `serialize(record)` - creates reference from record - `instantiate(string)` - deserializes reference - `preload_indexing_data(refs)` - batch loads DB records - `index` - returns `environment_specific_index_name('groups')` - `model_klass` - returns `::Group` - Implement instance methods: - `initialize(identifier, routing)` - `klass` - returns 'Group' - `serialize` - joins delimited components - `as_indexed_json` - builds the document hash Key fields to index: - id, name, path, full_name, full_path - description - parent_id, traversal_ids - visibility_level - avatar_url - created_at, updated_at - schema_version, type ### 3. Create Group Type Class **File**: `ee/lib/search/elastic/types/group.rb` Define the Elasticsearch index structure following `types/sbom/occurrence_ref.rb`: - `index_name` - delegates to Reference class - `target` - returns `::Group` - `mappings` - returns hash with `dynamic: 'strict'` and `properties: base_mappings` - `settings` - uses `Elastic::Latest::Config.settings` pattern - `base_mappings` private method with field definitions: - `type: { type: 'keyword' }` - `schema_version: { type: 'short' }` - `name: { type: 'text', index_options: 'positions', analyzer: :title_analyzer }` - `path: { type: 'keyword' }` - `full_name: { type: 'text', index_options: 'positions', analyzer: :my_ngram_analyzer }` - `full_path: { type: 'text', index_options: 'positions' }` - `description: { type: 'text', index_options: 'positions' }` - `traversal_ids: { type: 'keyword' }` - `visibility_level: { type: 'short' }` - Other fields as keyword/long/date types ### 4. Create Index Migration **File**: `ee/elastic/migrate/YYYYMMDDHHMMSS_create_groups_index.rb` ```ruby class CreateGroupsIndex < Elastic::Migration include ::Search::Elastic::MigrationCreateIndexHelper retry_on_failure def document_type :group end def target_class ::Group end end ``` **File**: `ee/elastic/docs/YYYYMMDDHHMMSS_create_groups_index.yml` ```yaml --- name: CreateGroupsIndex version: 'YYYYMMDDHHMMSS' description: Creates the groups Elasticsearch index group: group::global search milestone: 'X.Y' introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/XXXXXX obsolete: false marked_obsolete_by_url: marked_obsolete_in_milestone: ``` ### 5. Register in Helper **File**: `ee/lib/search/elastic/helper.rb` Add `::Group` to `ES_SEPARATE_CLASSES` array. ### 6. Update Delete Worker **File**: `ee/app/workers/elastic_delete_project_worker.rb` (if needed) Add `Group` to excluded_classes if it has special routing. ## Files to Create/Modify - `app/models/group.rb` or `ee/app/models/ee/group.rb` (add es_parent, preload_indexing_data) - `ee/lib/search/elastic/references/group.rb` (new) - `ee/lib/search/elastic/types/group.rb` (new) - `ee/elastic/migrate/YYYYMMDDHHMMSS_create_groups_index.rb` (new) - `ee/elastic/docs/YYYYMMDDHHMMSS_create_groups_index.yml` (new) - `ee/lib/search/elastic/helper.rb` (modify - add to ES_SEPARATE_CLASSES) ## Testing - `ee/spec/lib/search/elastic/references/group_spec.rb` - `ee/spec/lib/search/elastic/types/group_spec.rb` - `ee/spec/elastic/migrate/YYYYMMDDHHMMSS_create_groups_index_spec.rb` - Use `it_behaves_like 'migration creates a new index', TIMESTAMP, ::Group` - Test serialization/deserialization - Test as_indexed_json builds correct structure - Test preload_indexing_data ## Validation Steps (from MR !241326) 1. Create the index (Rails console): ```ruby rec = ::Elastic::DataMigrationService.migrations.find { |m| m.name == "CreateGroupsIndex" } rec.migrate rec.completed? # => true ``` 2. Verify index + mappings: ```bash curl -s "localhost:9200/gitlab-development-groups/_mapping?pretty" ``` 3. Index some records: ```ruby refs = ::Group.limit(10).map { |g| ::Search::Elastic::References::Group.serialize(g) } ::Elastic::ProcessInitialBookkeepingService.track!(*refs) ::Elastic::ProcessInitialBookkeepingService.new.execute ``` 4. Verify documents: ```bash curl -s -XPOST "localhost:9200/gitlab-development-groups/_refresh" curl -s "localhost:9200/gitlab-development-groups/_search?size=1&pretty" ``` ## Acceptance Criteria - [ ] Group model has es_parent and preload_indexing_data methods - [ ] Group Reference class created following MR !241326 pattern - [ ] Group Type class defines proper index mappings - [ ] Create index migration using MigrationCreateIndexHelper - [ ] Group registered in Helper::ES_SEPARATE_CLASSES - [ ] Tests follow existing patterns (shared examples) - [ ] Index can be created and documents indexed - [ ] Mappings verified in local ES cluster