Search: reindex outbox capture side

Description

Part of #2610

Incremental search indexing needs to know which artifact documents a data change touched. This MR adds the capture side only: a reindex_outbox table, a write-only ReindexOutboxTable with two enqueue helpers, and an enqueue call at every write path that affects the artifact search document, which is the tracked update-event approval flow plus the thirteen controller actions that write doc-relevant tables directly. Consuming the queue (claiming, cascade expansion, bulk indexing) is a separate follow-up MR.

Every enqueue runs inside the same database transaction as the data write it signals, so a change and its reindex signal commit or roll back together. The index can never learn about a change that never committed, and no committed change can lose its signal.

The table

It is not in the SQL dump, so apply it once per environment (idempotent):

docker exec -i cdlidev_mariadb_1 mariadb cdli_db < app/cake/config/schema/reindex_outbox.sql
CREATE TABLE IF NOT EXISTS reindex_outbox (
    id            BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
    entity_type   VARCHAR(50)  NOT NULL,
    entity_ids    JSON         NOT NULL,
    event_type    VARCHAR(50)  NOT NULL,
    metadata      JSON         NULL,
    status        ENUM('pending','processing','done','failed') NOT NULL DEFAULT 'pending',
    retry_count   TINYINT UNSIGNED NOT NULL DEFAULT 0,
    retry_after   DATETIME     NULL,
    created_at    DATETIME     NOT NULL DEFAULT CURRENT_TIMESTAMP,
    locked_at     DATETIME     NULL,
    processed_at  DATETIME     NULL,
    error         TEXT         NULL,
    KEY idx_claim (status, retry_after, created_at)
);

Tests don't need that manual step: tests/bootstrap.php creates the table on the test connection at phpunit boot from the same .sql, and enables transactional fixture isolation (TransactionStrategy) so any test that declares a fixture rolls back everything it writes. That isolation matters because the test datasource points at the live dev database. The state columns (status, retry_*, locked_at, processed_at, error) exist from day one so the poller needs no later schema change, but nothing in this MR reads or sets them past the DB defaults.

Row shapes

event_type entity_ids metadata
direct_change ≤500 artifact ids per row null
cascade_<table> [], resolved lazily by the poller at claim time {entity_id, op}
cascade_external_resources (delete) ≤500 pre-captured ids per row {entity_id, op: delete} on every row

enqueue() sanitizes (numeric, positive, deduplicated), chunks at 500 ids per row, and treats an empty list as a successful no-op. entity_ids and metadata use Cake's json type (MariaDB stores JSON as LONGTEXT), so consumers get arrays back.

Capture points

The tracked path is one line inside approve()'s existing transaction, after apply() and before commit. It calls a helper that enqueues three things:

  1. the direct artifact set post-apply (new artifacts carry real ids by then),
  2. the artifacts behind annotation-only approvals, resolved from the annotations' artifact_asset_id column (apply reloads annotations without their nested asset association, so the direct set alone would miss them), and
  3. one lazy cascade row per doc-affecting entities_updates row (11 entity tables; places is excluded because the document doesn't read it, a JSON-null update means delete).

A thrown or false enqueue takes the method's existing failure path: rollback, no signal, no data change.

The direct write paths each wrap their existing save or delete plus the enqueue in a transaction, keyed off the same success flag the action already used:

  • visual assets add/edit/delete/update-metadata (edit enqueues only on a successful save, its unconditional redirect is unchanged)
  • inscription atf2conll resolution
  • chemical-data delete
  • the seal-chemistry CSV upload (multi-artifact, deduplicated; the duplicate-upload PDOException handler now rolls back too)
  • external-resource edit (a base_url change fans out to every linked artifact, so it is a lazy cascade) and delete (affected artifact ids are pre-captured before the junction rows DB-cascade away)
  • author admin edit/delete and self-service edit
  • journal edit/delete, archive edit, region edit
  • update-event edit (enqueues the event's resolved artifact set only when the event is already approved, since only then is its update block part of any document, the edit itself stays unguarded)

Tests

37 new tests (@group db, local-only like the existing document-builder tests, seeds use ids ≥ 900,000,000 inside the rolled-back fixture transaction). CI runs phpcs and phpstan only, not phpunit, and the bare suite aborts on three pre-existing legacy files, so run the new tests path-scoped:

vendor/bin/phpunit tests/TestCase/Model/Table/ReindexOutboxTableTest.php \
  tests/TestCase/Controller/UpdateEventsControllerTest.php \
  tests/TestCase/Controller/AuthorsControllerTest.php \
  tests/TestCase/Controller/Admin
Edited by sung

Merge request reports

Loading