### Problem to solve Currently, Advanced Search only supports searching wikis on a project level. Our indexing code doesn't have support for repositories that belongs to groups (a wiki is a repository under the hood). Customers are confused when they search for wikis in a group expecting to see group wikis, but they only get project wikis results. ### Intended users Customers who use and create wikis on a group level. ### Proposal When users search within a group only then wikis related to the selected group and wikis related to all the projects inside this group will be shown. <!-- How are we going to solve the problem? Try to include the user journey! https://about.gitlab.com/handbook/journeys/#user-journey --> Make group-level wikis searchable. project-level wikis are stored in the main index. There may not be any advantage to using that index since group-level wikis are not associated with a project. Permissions will need to be propagated into the documents Technical steps: Consideration to move all project wikis to their own `wiki` index as a first step. https://gitlab.com/groups/gitlab-org/-/epics/6979 1. ~backend Advanced Search migration to create a new index for all wikis and backfill Project wikis over to new index (calling the indexer worker with some flag to trigger it). We would need to index in both old/new indexes until the backfill is completed. Add hashed namespace_id to the new index. The new index mappings should support both project and group wikis. Do not think we should use the default routing, we would hit all shards in the index for a specific top level namespace. For new index, we should use routing of `group_<id>` (not top level namespace) for `GroupWiki` and `project_<id>` for `ProjectWiki`. The routing should **only be used** when performing **project** scoped searches, it cannot be used for **group** or **global** searches. We cannot use the routing when searching on a parent group, because it would miss results (we are not using top level namespace so we may miss descendant results) 2. ~backend Add functionality to the [gitlab-elasticsearch-indexer](https://gitlab.com/gitlab-org/gitlab-elasticsearch-indexer) to index group level wikis. * The command line flag `--project-path` should be deprecated (and scheduled for removal in the next major milestone). * A new command line flag `--path` should be used in it's place to support group and project level indexing of wiki/repositories. * Change `project_id` to a named argument (currently it is an argument) and should be made optional. * Add `group_id` as a named argument, maybe this should be required (all data has an associated group_id, even repositories and project-wikis through the project) * Consider `traversal_ids` (formerly known as `namespace_ancestry`) optimization at time of indexing (an optional argument). group_id could also be pulled from this argument. * ~~We might consider adding a new `--blob-type`, could be used to determine which index to send to and how to build the document up~~ (will not need this since we are going to use one index) 3. ~backend Support indexing for group wikis * create new `WikiInstanceProxy` and `WikiClassProxy` classes, these will eventually replace the existing `ProjectWikiInstanceProxy` and `ProjectWikiClassProxy` but they cannot be removed until the next major release (there will be a migration completed guard to determine which to use). They should be marked as deprecated with a link to an issue with a milestone of 17.0 * updates for `ElasticCommitIndexerWorker` * use `options` to pass `group_id` and need to adjust the code to not return if project_id is nil * updates for `Gitlab::Elastic::Indexer` * need to pass group to the elasticsearch_indexer and it will also pass the new arguments for index name to the elasticsearch_indexer 4. ~backend Support for removal from index when a group is deleted (like `ElasticProjetDeleteWorker` but for groups) * `ElasticProjectDeleteWorker` needs to be updated to delete from new wiki index * `ElasticGroupDeleteWorker` needs to be created 5. ~backend Support searching for group wikis, behind ~"feature flag" 6. ~backend Advanced Search migration to backfill data into index 7. ~backend Add support for searching via APIs 8. ~frontend Support searching for group wikis: https://gitlab.com/gitlab-org/gitlab/-/issues/385593 9. add any deprecation issues required and set to milestone of 17.0 --- ## Progress status ### Indexer 1. Add `wiki-access-level` & `WikiPermissions` -> [!175](https://gitlab.com/gitlab-org/gitlab-elasticsearch-indexer/-/merge_requests/175) :white_check_mark: 2. Rename `--project-path` to `--path` -> [!176](https://gitlab.com/gitlab-org/gitlab-elasticsearch-indexer/-/merge_requests/176) :white_check_mark: 3. Replace argument with `--project-id` -> [!178](https://gitlab.com/gitlab-org/gitlab-elasticsearch-indexer/-/merge_requests/178) :white_check_mark: 3. Release intermediate indexer version -> [!180](https://gitlab.com/gitlab-org/gitlab-elasticsearch-indexer/-/merge_requests/180) :white_check_mark: 4. Separate index for wikis (+ add `group_id`) -> [!174](https://gitlab.com/gitlab-org/gitlab-elasticsearch-indexer/-/merge_requests/174) :white_check_mark: ### GitLab 1. Introduce `Elastic::GroupIndexStatus` to support group level wikis -> https://gitlab.com/gitlab-org/gitlab/-/merge_requests/107579 :white_check_mark: 1. Update `Gitlab::Elastic::Indexer` to support new parameters (`--full-path` and `--project_id`) & update indexer version -> https://gitlab.com/gitlab-org/gitlab/-/merge_requests/107668 :white_check_mark: 1. Add a migration to add project permissions to wiki documents -> https://gitlab.com/gitlab-org/gitlab/-/merge_requests/116944 :white_check_mark: 1. Add indexing for project wikis to the separate index and add searching for project wiki -> https://gitlab.com/gitlab-org/gitlab/-/merge_requests/107365 :white_check_mark: 1. Add indexing for group wikis -> https://gitlab.com/gitlab-org/gitlab/-/merge_requests/120171 :white_check_mark: 1. Migrate existing Group Wikis to Elasticsearch -> https://gitlab.com/gitlab-org/gitlab/-/merge_requests/122529 :white_check_mark: 1. Add searching logic for the GroupWikis -> https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123522 :white_check_mark: 1. Update [documentation](https://docs.gitlab.com/ee/user/project/wiki/group.html -> https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123522 :white_check_mark: 1. Delete `blob_wiki` documents from the main index -> https://gitlab.com/gitlab-org/gitlab/-/merge_requests/126538 ### Release notes The group-level wiki will be included in Advanced global and group wiki searches.