## Objective Add OpenSearch as a second search engine running alongside the existing Elasticsearch, without removing or disrupting ES. Both engines run simultaneously so we can compare them side by side. The app can query either one based on a URL parameter. --- ## Current Setup ### Elasticsearch service (docker-compose.dev.yml:147-178) ```yaml elasticsearch: image: docker.elastic.co/elasticsearch/elasticsearch:7.17.28 hostname: elasticsearch environment: - discovery.type=single-node - bootstrap.memory_lock=true - xpack.security.enabled=false - "ES_JAVA_OPTS=-Xms2g -Xmx2g" - "ELASTIC_PASSWORD=elasticchangeme" ports: - '127.0.0.1:9200:9200' - '127.0.0.1:9300:9300' ``` ### ES URL in app.php (line 465) ```php 'elasticsearch' => 'http://elastic:elasticchangeme@elasticsearch:9200', ``` ### How the app talks to ES `ElasticSearchQuery.php` reads the URL via `Configure::read('ServiceUrls.elasticsearch')` and makes raw HTTP POST calls using `Cake\Http\Client`. No ES SDK, no library. Just HTTP. There are 3 call sites: - `ElasticSearchQuery::all()` (line 251) - `POST /_search` - `ElasticSearchQuery::countAll()` (line 752) - `POST /_count` - `Admin/SearchController::view()` (line 23) - `POST /_search` (bypasses ElasticSearchQuery) ### Logstash output (logstash.conf:187-197) Writes to `elasticsearch:9200`. Stays pointing at ES for now since we're not removing it. --- ## API Compatibility Research OpenSearch was forked from Elasticsearch OSS 7.10.2. Our ES is 7.17.28. Key findings: | Feature | Compatible? | Notes | |---------|-------------|-------| | `_search` endpoint | Yes | Same request/response format | | `_count` endpoint | Yes | No differences | | `_bulk` endpoint | Yes | Document IDs must be \<= 512 bytes (not an issue for us, we use integer IDs) | | Query DSL (bool, match, term, range, nested) | Yes | Core DSL is the same | | `missing` param in terms aggregations | Yes | Used in ElasticSearchQuery.php:174, works in OpenSearch | | Index mappings (text, keyword, nested, geo_point) | Yes | Standard field types are the same | | Basic auth format (`http://user:pass@host:port`) | Yes | OpenSearch default dev credentials are `admin:admin` | | `search_after` pagination | Yes | Used in ElasticSearchApiPaginator | **Risk area:** ES 7.17 has features added in 7.11-7.17 that may not exist in OpenSearch 2.x (since it forked from 7.10). We need to test all existing queries against OpenSearch to catch anything that breaks. From reading the codebase, the query patterns used in `ElasticSearchQuery.php` stick to core features (bool, match, term, range, nested, aggregations) that are all present in both. --- ## Approach ### 1. Add OpenSearch service to docker-compose.dev.yml ```yaml # Image--infrastructure:opensearch opensearch: image: opensearchproject/opensearch:2.19.5 hostname: opensearch restart: on-failure environment: - cluster.name=os-cluster - node.name=opensearch - discovery.type=single-node - bootstrap.memory_lock=true - DISABLE_SECURITY_PLUGIN=true # no SSL/auth overhead in dev - "OPENSEARCH_JAVA_OPTS=-Xms2g -Xmx2g" # renamed from ES_JAVA_OPTS ulimits: nofile: 65536 memlock: soft: -1 hard: -1 volumes: - type: bind source: ./data/opensearch/data target: /usr/share/opensearch/data ports: - '127.0.0.1:9202:9200' # different external port to avoid conflict with ES - '127.0.0.1:9302:9300' networks: - nodelocal-private depends_on: - mariadb ``` Key differences from the ES service: - `OPENSEARCH_JAVA_OPTS` instead of `ES_JAVA_OPTS` - `DISABLE_SECURITY_PLUGIN=true` instead of `xpack.security.enabled=false` - External ports 9202/9302 to avoid conflict (internally still 9200/9300) - Separate data directory (`./data/opensearch/data`) via bind mount - Different cluster name (`os-cluster`) so they don't try to join each other ### 2. OpenSearch Dashboards (Kibana equivalent) - skipping for now We're not adding OpenSearch Dashboards. Running both ES and OpenSearch already doubles the JVM memory usage (2GB each). Adding Dashboards would mean another service on top of that. We can query OpenSearch directly via `curl` or through the `?engine=opensearch` URL parameter once that's implemented. If we need Dashboards later for debugging, it's easy to add. ### 3. Add OpenSearch URL to app.php ```php 'ServiceUrls' => [ // ... existing URLs ... 'elasticsearch' => 'http://elastic:elasticchangeme@elasticsearch:9200', 'opensearch' => 'http://opensearch:9200', // NEW - no auth needed (security disabled) ], ``` With `DISABLE_SECURITY_PLUGIN=true`, OpenSearch doesn't require authentication, so no `admin:admin@` prefix is needed. If we enable security later, the URL would change to `https://admin:admin@opensearch:9200`. ### 4. Add data directory for OpenSearch ```bash mkdir -p dev/data/opensearch/data ``` And add to `.gitignore`: ``` dev/data/opensearch/data/ ``` ### 5. No volume definition needed The OpenSearch service uses a bind mount (matching the same pattern as the existing ES service), so there's no need to add a named volume to the `volumes:` section. ### 6. Verify OpenSearch is working After `docker compose up`: ```bash # Check ES is still running curl http://localhost:9200 # Should return: {"name": "elasticsearch", "version": {"number": "7.17.28"}, ...} # Check OpenSearch is running curl http://localhost:9202 # Should return: {"name": "opensearch", "version": {"number": "2.19.5", "distribution": "opensearch"}, ...} ``` ### 7. Load the same index mapping into OpenSearch The existing `artifacts.json` mapping template needs to be loaded into OpenSearch too. This can be done via a curl command or through the full reindex command later: ```bash # Load the mapping template into OpenSearch curl -X PUT "http://localhost:9202/_template/artifacts" \ -H "Content-Type: application/json" \ -d @dev/data/logstash/resources/artifacts.json ``` --- ## What Does NOT Change | Component | Change? | Why | |-----------|---------|-----| | `ElasticSearchQuery.php` | No changes | It reads the URL from config. The query DSL is compatible with both engines. | | Logstash | No changes | Keeps writing to ES as it does today | | Existing search pages | No changes | They keep querying ES by default | | `Admin/SearchController.php` | No changes yet | Will be updated when we implement the `?engine=` parameter | | Index mappings (`artifacts.json`) | No changes | Same mapping works on both ES and OpenSearch | --- ## What Comes Later (not in this issue) These are handled by other sub-issues: - **`?engine=opensearch` URL parameter** for side-by-side comparison (search inconsistency issue) - **Poller writing to both ES and OpenSearch** (incremental indexing issue) - **Full reindex command populating the OpenSearch index** (fallback utilities issue) - **Mapping improvements** like removing `ignore_above: 256` on comment fields (separate task) This issue is only about getting OpenSearch running alongside ES and verifying it works. --- ## Risks and Compatibility Notes ### 1. ES 7.11-7.17 features may not exist in OpenSearch OpenSearch forked from ES 7.10. Any features added to ES after that (7.11 through 7.17) are not guaranteed to exist in OpenSearch. From reading the codebase, `ElasticSearchQuery.php` uses standard features (bool, match, term, range, nested, terms aggregations, search_after) that are all present in both. But we need to run the full test suite against OpenSearch to confirm. ### 2. Security plugin fallback If `DISABLE_SECURITY_PLUGIN=true` doesn't work on the specific OpenSearch version (there were reports of issues in 2.12), the fallback is to add this to an `opensearch.yml` config file: ```yaml plugins.security.disabled: true ``` ### 3. Memory usage Running both ES and OpenSearch locally means double the JVM memory. Each is configured with 2GB (`-Xms2g -Xmx2g`). Developers with limited RAM may want to reduce one or both to 1GB, or only start one at a time: ```bash docker compose up -d elasticsearch # only ES docker compose up -d opensearch # only OpenSearch docker compose up -d # both ``` ### 4. Internal vs external ports Inside the Docker network, both services listen on port 9200. Other containers reach them by hostname: - `http://elasticsearch:9200` (ES, used by existing code) - `http://opensearch:9200` (OpenSearch, used by new code) External access from the host machine uses different ports: - `http://localhost:9200` (ES) - `http://localhost:9202` (OpenSearch) --- ## Estimate | | Time | |--|------| | Development | \~2 days. Add the Docker service, create data directory, update app.php config, load mapping template, update .gitignore. | | Testing | \~1 day. Verify both services start, verify API responses, manually run a few search queries against OpenSearch to confirm compatibility, check memory usage. | | **Total** | **\~3 days** | --- ## Acceptance Criteria - [x] OpenSearch 2.19.5 service added to docker-compose.dev.yml alongside existing ES - [x] OpenSearch starts successfully with `docker compose up` - [x] Security plugin disabled for local dev - [x] OpenSearch URL added to `ServiceUrls` in app.php - [x] Data directory created and gitignored - [x] `curl http://localhost:9202` returns OpenSearch version info - [x] Artifacts mapping template loaded into OpenSearch - [x] Existing Elasticsearch service still works exactly as before (no regressions) - [x] Both services accessible from other containers via hostname on the Docker network