Search: reindex outbox capture side
Description
Part of #2610
Incremental search indexing needs to know which artifact documents a data change touched. This MR adds the capture side only: a reindex_outbox table, a write-only ReindexOutboxTable with two enqueue helpers, and an enqueue call at every write path that affects the artifact search document, which is the tracked update-event approval flow plus the thirteen controller actions that write doc-relevant tables directly. Consuming the queue (claiming, cascade expansion, bulk indexing) is a separate follow-up MR.
Every enqueue runs inside the same database transaction as the data write it signals, so a change and its reindex signal commit or roll back together. The index can never learn about a change that never committed, and no committed change can lose its signal.
The table
It is not in the SQL dump, so apply it once per environment (idempotent):
docker exec -i cdlidev_mariadb_1 mariadb cdli_db < app/cake/config/schema/reindex_outbox.sqlCREATE TABLE IF NOT EXISTS reindex_outbox (
id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
entity_type VARCHAR(50) NOT NULL,
entity_ids JSON NOT NULL,
event_type VARCHAR(50) NOT NULL,
metadata JSON NULL,
status ENUM('pending','processing','done','failed') NOT NULL DEFAULT 'pending',
retry_count TINYINT UNSIGNED NOT NULL DEFAULT 0,
retry_after DATETIME NULL,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
locked_at DATETIME NULL,
processed_at DATETIME NULL,
error TEXT NULL,
KEY idx_claim (status, retry_after, created_at)
);Tests don't need that manual step: tests/bootstrap.php creates the table on the test connection at phpunit boot from the same .sql, and enables transactional fixture isolation (TransactionStrategy) so any test that declares a fixture rolls back everything it writes. That isolation matters because the test datasource points at the live dev database. The state columns (status, retry_*, locked_at, processed_at, error) exist from day one so the poller needs no later schema change, but nothing in this MR reads or sets them past the DB defaults.
Row shapes
event_type |
entity_ids |
metadata |
|---|---|---|
direct_change |
≤500 artifact ids per row | null |
cascade_<table> |
[], resolved lazily by the poller at claim time |
{entity_id, op} |
cascade_external_resources (delete) |
≤500 pre-captured ids per row | {entity_id, op: delete} on every row |
enqueue() sanitizes (numeric, positive, deduplicated), chunks at 500 ids per row, and treats an empty list as a successful no-op. entity_ids and metadata use Cake's json type (MariaDB stores JSON as LONGTEXT), so consumers get arrays back.
Capture points
The tracked path is one line inside approve()'s existing transaction, after apply() and before commit. It calls a helper that enqueues three things:
- the direct artifact set post-apply (new artifacts carry real ids by then),
- the artifacts behind annotation-only approvals, resolved from the annotations'
artifact_asset_idcolumn (apply reloads annotations without their nested asset association, so the direct set alone would miss them), and - one lazy cascade row per doc-affecting
entities_updatesrow (11 entity tables;placesis excluded because the document doesn't read it, a JSON-null update means delete).
A thrown or false enqueue takes the method's existing failure path: rollback, no signal, no data change.
The direct write paths each wrap their existing save or delete plus the enqueue in a transaction, keyed off the same success flag the action already used:
- visual assets add/edit/delete/update-metadata (edit enqueues only on a successful save, its unconditional redirect is unchanged)
- inscription atf2conll resolution
- chemical-data delete
- the seal-chemistry CSV upload (multi-artifact, deduplicated; the duplicate-upload PDOException handler now rolls back too)
- external-resource edit (a
base_urlchange fans out to every linked artifact, so it is a lazy cascade) and delete (affected artifact ids are pre-captured before the junction rows DB-cascade away) - author admin edit/delete and self-service edit
- journal edit/delete, archive edit, region edit
- update-event edit (enqueues the event's resolved artifact set only when the event is already approved, since only then is its update block part of any document, the edit itself stays unguarded)
Tests
37 new tests (@group db, local-only like the existing document-builder tests, seeds use ids ≥ 900,000,000 inside the rolled-back fixture transaction). CI runs phpcs and phpstan only, not phpunit, and the bare suite aborts on three pre-existing legacy files, so run the new tests path-scoped:
vendor/bin/phpunit tests/TestCase/Model/Table/ReindexOutboxTableTest.php \
tests/TestCase/Controller/UpdateEventsControllerTest.php \
tests/TestCase/Controller/AuthorsControllerTest.php \
tests/TestCase/Controller/Admin