docs: add ADR 023 and the indexing engine docs
What does this MR do and why?
This adds ADR 023. It proposes one self-contained engine for all indexing work: a flow is a planner and a worker, and the engine owns lanes, retries, scaling, a ledger and migrations. After review, the ADR keeps only the decisions. A new architecture page explains how the engine works. A new guide, "Extending the indexing engine", shows the API. ADR 017 gets an "Amended by" note.
Why this choice
Orbit Next and third-party data need new flows and new ontology versions at runtime, with no engine change. The ADR stays short and does not change after it merges. The architecture page and the guide change with the code.
Related Issues
Relates to orbit#21
Testing
mise lint:docs passes, including the link check.
Performance Analysis
- This merge request does not introduce any performance regression. If a performance regression is expected, explain why.
Agent context: long-form analysis, file-by-file walkthroughs, profiler output, alternatives considered
- Start with the Decision section of
docs/pages/engineering/adr/023_self_contained_indexing_engine.md. It lists the eight decisions on one page. - Then read
docs/pages/engineering/architecture/indexing/indexing-engine.md. It follows one push through the engine and lists what can go wrong. - Then read
docs/pages/engineering/development/extending-the-indexing-engine.md. It has the API, the Helm config and the ledger DDL. - End with the Status line in
017_clone_based_non_blocking_migrations.mdand the twoREADME.mdlinks.
The diagram in docs/assets/adr/ is an SVG export of the Excalidraw drawing in the epic.