Draft: Add OpenSearch Serverless support for Advanced Search

What does this MR do and why?

This MR adds support for AWS OpenSearch Serverless to GitLab's Advanced Search feature by introducing a new application setting that allows administrators to choose between regular OpenSearch and OpenSearch Serverless.

When OpenSearch Serverless is enabled, the Elasticsearch client uses aoss as the service name for AWS SigV4 request signing instead of es, as required by AWS OpenSearch Serverless.

Related Issue

Closes #575737

Implementation Details

Based on the updated diff, here's the revised Implementation Details section:

Implementation Details

The implementation follows the same pattern as the elasticsearch_client_adapter setting by using the existing JSONB elasticsearch column:

1. JSON Schema (ee/app/validators/json_schemas/application_setting_elasticsearch.json):

   • Added elasticsearch_aws_serverless boolean field with default false

2. Application Setting Model (ee/app/models/ee/application_setting.rb):

   • Added elasticsearch_aws_serverless as a jsonb_accessor attribute
   • Included aws_serverless in the elasticsearch_config hash

3. Elastic Client (ee/lib/gitlab/elastic/client.rb):

   • Added constants: AWS_SERVICE_NAME_SERVERLESS = 'aoss' and AWS_SERVICE_NAME_REGULAR = 'es'
   • Created aws_service_name(config) method to return the appropriate service name
   • Updated AWS SigV4 signing to use dynamic service name based on configuration
   • Applied ServerlessClientPatch to the client when serverless is enabled

4. Serverless Client Patch (ee/lib/search/elastic/serverless_client_patch.rb):

   • New module that patches the Elasticsearch client to handle OpenSearch Serverless API limitations
   • Intercepts delete_by_query and converts to search + bulk delete fallback when not supported
   • Provides stub implementations for indices.stats, cluster.stats, cluster.health, info, and ping
   • Handles get_mapping NotFound errors gracefully

5. Elastic Indexer (ee/lib/gitlab/elastic/indexer.rb):

   • Sets AWS_SERVICE_NAME environment variable based on serverless configuration

6. Admin Helper (ee/app/helpers/ee/application_settings_helper.rb):

   • Added :elasticsearch_aws_serverless to visible attributes

7. Admin UI (ee/app/views/admin/application_settings/_elasticsearch_form.html.haml):

   • Added checkbox in AWS OpenSearch IAM credentials section with help text

8. Index Configuration (ee/lib/elastic/latest/config.rb):

   • Updated codec selection to use zstd for serverless, best_compression for regular OpenSearch

9. Active Context OpenSearch Client (gems/gitlab-active-context/lib/active_context/databases/opensearch/client.rb):

   • Added aws_service_name method to support serverless configuration

10. Tests:

    • Added comprehensive tests for AWS service name selection with serverless enabled/disabled
    • Added tests for the ServerlessClientPatch module covering delete_by_query fallback, info, ping, indices, and cluster methods
    • Added tests for indexer AWS service name environment variable
    • Added tests for active context OpenSearch client

11. Documentation:

    • Updated API documentation (doc/api/settings.md)
    • Updated Elasticsearch admin documentation (doc/integration/advanced_search/elasticsearch.md)
    • Added translation strings (locale/gitlab.pot)

Benefits

• ✅ No database migration required (uses existing JSONB column)
• ✅ Backward compatible (defaults to false)
• ✅ Follows established patterns in the codebase
• ✅ Simple checkbox UI for administrators
• ✅ Validated by JSON schema
• ✅ Comprehensive test coverage

How to set up and validate locally

1. Enable Elasticsearch for GDK following the Elasticsearch setup guide
2. Navigate to Admin Area > Settings > Advanced Search
3. Expand the Advanced search section
4. Scroll to the AWS OpenSearch IAM credentials section
5. Check the "Use AWS OpenSearch Serverless" checkbox
6. Save the settings
7. Verify that the Elasticsearch client uses aoss as the service name when making AWS requests

Screenshots

Admin UI

The new checkbox appears in the AWS OpenSearch IAM credentials section:

Test

With the option selected

➜  gitlab git:(575737-opensearch-serverless-support) bundle exec rake gitlab:elastic:create_empty_index
Index 'gitlab-development-notes-20251119-0204' has been created.
Alias 'gitlab-development-notes' -> 'gitlab-development-notes-20251119-0204' has been created.
Index 'gitlab-development-merge_requests-20251119-0204' has been created.
Alias 'gitlab-development-merge_requests' -> 'gitlab-development-merge_requests-20251119-0204' has been created.
Index 'gitlab-development-commits-20251119-0204' has been created.
Alias 'gitlab-development-commits' -> 'gitlab-development-commits-20251119-0204' has been created.
Index 'gitlab-development-users-20251119-0204' has been created.
Alias 'gitlab-development-users' -> 'gitlab-development-users-20251119-0204' has been created.
Index 'gitlab-development-wikis-20251119-0204' has been created.
Alias 'gitlab-development-wikis' -> 'gitlab-development-wikis-20251119-0204' has been created.
Index 'gitlab-development-projects-20251119-0204' has been created.
Alias 'gitlab-development-projects' -> 'gitlab-development-projects-20251119-0204' has been created.
Index 'gitlab-development-work_items-20251119-0204' has been created.
Alias 'gitlab-development-work_items' -> 'gitlab-development-work_items-20251119-0204' has been created.
Index 'gitlab-development-vulnerabilities-20251119-0204' has been created.
Alias 'gitlab-development-vulnerabilities' -> 'gitlab-development-vulnerabilities-20251119-0204' has been created.
Index '{"gitlab-development-20251119-0204"=>"gitlab-development"}' has been created.
Alias 'gitlab-development' → '{"gitlab-development-20251119-0204"=>"gitlab-development"}' has been created

Without the option selected

➜  gitlab git:(575737-opensearch-serverless-support) ✗ bundle exec rake gitlab:elastic:create_empty_index
rake aborted!
Elasticsearch::Transport::Transport::Errors::Forbidden: [403]
/Users/changzhengliu/gitlab/gitlab-development-kit/gitlab/ee/lib/gitlab/elastic/helper.rb:230:in `index_exists?'
/Users/changzhengliu/gitlab/gitlab-development-kit/gitlab/ee/lib/gitlab/elastic/helper.rb:432:in `create_index'
/Users/changzhengliu/gitlab/gitlab-development-kit/gitlab/ee/lib/gitlab/elastic/helper.rb:206:in `create_empty_index'
/Users/changzhengliu/gitlab/gitlab-development-kit/gitlab/ee/app/services/search/rake_task_executor_service.rb:104:in `create_empty_index'
/Users/changzhengliu/gitlab/gitlab-development-kit/gitlab/ee/app/services/search/rake_task_executor_service.rb:71:in `execute'
/Users/changzhengliu/gitlab/gitlab-development-kit/gitlab/ee/lib/tasks/gitlab/elastic.rake:92:in `block (3 levels) in <main>'
/Users/changzhengliu/.local/share/mise/installs/ruby/3.3.9/bin/bundle:36:in `<main>'
Tasks: TOP => gitlab:elastic:create_empty_index
(See full trace by running task with --trace)

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.

Changelog: added EE: true