(Size L) Cells 1.5: Advanced search deployment strategy

Problem to solve

Advanced search on GitLab.com currently uses Elasticsearch hosted in ElasticCloud.

GET currently supports OpenSearch on AWS.

Having two different search backends across Cells will introduce issues with the divergent paths:

Differences in search behavior

While Elasticsearch and OpenSearch are close. Not all index settings and mappings are the same. #454764 (closed) details work where divergent paths in search backends are being handled

Automation of upgrade and index maintenance tasks

We plan to move towards automation of all upgrades and index maintenance tasks. Maintenance tasks are usually done manually using change request issues. The automation of index maintenance will use the Advanced search migration framework and ensure those migrations only run for GitLab.com to not impact self managed instances. Having a different search backend across cells may mean different migrations would need to be written for different cells.

Supporting cross cluster search

While we do not currently do cross cluster search today. We've been discussing using it to support global project search in cells. Cross cluster search is not possible across OpenSearch and Elasticsearch backends

Limitations for solutions in Org Mover

Using two different search backends would remove the ability to use Elasticsearch APIs to move organization level Advanced search data. We would need to rely upon the existing indexing pipeline which impacts Gitaly and PostgreSQL

Proposal

Use Elasticsearch for Cells with the same version as GitLab.com

Option 1 - Add support for cell specific indexes to GitLab

Keep the Elasticsearch cluster as one cluster (hosted in ElasticCloud) but allow each cell to prefix index names so they are separated.

Option 2 - Add support for Elasticsearch to GET

Each cell would have it's own cluster.