Search: index lifecycle in PHP (versioned indexes, alias swap, artifacts mapping)

No PHP code creates a search index today. The Logstash template owns the artifacts mapping, and the pipeline pushes into whatever that template made. The template goes away at cutover, so the mapping has to move into the application first. This MR adds the machinery for creating and promoting indexes. The artifacts mapping is its first consumer.

Two aliases per entity, one index. SearchAliasRegistry gains writeAliasFor() and indexName(entityType, version). Searches use the read alias, artifacts. The poller, a backfill and the reconcile check use the write alias, artifacts_write. Concrete indexes are artifacts_v1, artifacts_v2 and so on.

Both aliases normally point at the same index. They only separate during a rebuild. The write alias moves to the new index first, so edits made while it fills land there. The read alias moves last, once the index is complete. A search never sees a half-filled index.

I also considered one alias over both indexes with is_write_index. Writes route correctly, but a search then returns every backfilled artifact twice for the whole rebuild. Duplicates are worse than stale reads.

DocumentBuilderInterface::indexName() is removed. A builder can no longer say which index it writes to. With two aliases that is not a property of the document. It depends on who is doing the writing, and both callers already know. ReindexProcessor asks the registry for the write alias of the row's entity type. ReindexReconcileCommand uses the same name, since it checks whether pipeline writes landed. IndexNameAgreementTest is deleted. ReindexProcessorTest now asserts the write target through a stub registry instead.

The artifacts mapping is a PHP class. ArtifactIndexMapping declares all top-level fields and all nested inner fields. The names are the document builder's, not the legacy template's. The template has three Logstash fields the builder never emits. Four more sit under older names. Three that the builder emits are missing from it. All seven names the builder uses that the template lacks currently land by dynamic mapping.

I chose a class over a JSON file so the reasoning can sit next to each value as a comment.

Three departures from the template, each measured on the live corpus and both engines:

• The five wildcard sub-fields become keyword. wildcard is Elasticsearch-only. On real data keyword answered 36 of 36 probe queries identically.
• ignore_above is 10922 everywhere, up from 256. The bound counts UTF-16 code units, worst case 3 bytes each, so 10922 * 3 is exactly Lucene's 32766-byte term limit. That is the largest bound that can never reject a document. Values the old bound truncated become matchable again, a few hundred annotation rows and fewer in genre_comments, cdli_comments and artifact_comments.
• All seven nested containers are declared, even where no artifact has data yet. A nested query against an undeclared path is a hard error on both engines.

dynamic stays on, with a dynamic template so an unexpected string still gets a bounded keyword. date_detection is off, so free text like dates_referenced cannot be typed as a date from one parseable value. No analyzers or normalizers. Diacritic folding is out of scope here, and changing analysis means a new version and a reindex, which this MR makes cheap.

bin/cake search_index_create --entity artifact creates the next version, or an explicit --version. It then reads GET /<index>/_mapping back and compares it leaf by leaf, both directions. The create response only confirms the request was accepted, which is not the same thing.

The read-back also catches a trap. Elasticsearch still carries the legacy artifacts template at index_patterns: ["*"], so it merges those fields into any new index there. Pointed at ES, the command fails and names all fifteen merged leaves. A failed index is left in place to inspect. Re-running without a version creates the next one. No alias is touched.

bin/cake search_index_swap --entity artifact --version N --alias read|write|both moves aliases in one _aliases call, removes and adds together. No reader sees an alias unbound or spanning two indexes. --alias has no default. Passing both mid-rebuild would put searches on a half-filled index, so it has to be asked for by name. Re-running is a no-op.

Both commands are thin wrappers over IndexLifecycleService, matching the one-class-per-operation shape of the four reindex_* commands.

Two things to watch when running it. The read swap is what finishes a rebuild. Between the two swaps, reconcile audits the new index while search still serves the old one. A forgotten read swap therefore looks healthy while the site serves stale results. Second, on a fresh machine, create and swap before starting the poller. A push through an unbound write alias auto-creates a junk index under that name. The same hazard exists today with artifacts.

Not in this MR. No backfill, so the new index stays empty until the reindex command lands. No delete-old-version tooling. No mappings for the other four entities. The read path is untouched and still queries Elasticsearch through the read alias.

Testing

The mapping tests restate the spec. Every name the builder emits has a mapping entry, checked against the parity fixtures, and nothing is mapped that the builder never emits. Every field must match the standard text-plus-keyword shape, one of the listed non-text exceptions, or a nested container following the same rule. A wrong type or a stray ignore_above: 256 anywhere fails.

The lifecycle service is tested against a mocked client. That covers the create-and-verify sequence and each way a read-back can disagree, including the template-contamination case. Command tests cover option validation and version defaulting through a service seam, so nothing reaches an engine.

SearchIndexE2ETest runs the real thing against local OpenSearch. It creates v1, checks the stored mapping, binds both aliases, writes through the write alias and reads back through the read alias. Then it creates v2 and moves the write alias while reads stay on v1. It refuses to run if any artifacts* name exists, and deletes both versions afterwards.