feat(schema): serve requests from the active ontology archive

What does this MR do and why?

Queries follow the active ontology archive through promotion and rollback. Requests already running keep their original snapshot; if loading the active archive fails, new schema-dependent requests are rejected until recovery.

How the whole flow works

Serving continues during backfill. Blue shows this MR's changes.

%%{init: {"themeVariables": {"fontSize": "13px"}, "flowchart": {"nodeSpacing": 16, "rankSpacing": 20, "padding": 8}}}%%
flowchart LR
    subgraph Upgrade[Prepare the next schema]
        direction TB
        Start[New release starts] --> Archives[Publish new archive<br/>Restore supported missing active archive]
        Archives -->|Failure| Stop[Stop migration<br/>Keep active version]
        Archives -->|Ready| Tables[Prepare target tables]
        Tables --> Backfill[Rebuild changed data]
        Backfill --> Ready{Required data ready<br/>and target archive usable?}
        Ready -->|Not yet| Backfill
        Ready -->|Yes| Promote[Make new version active]
    end

    subgraph Serving[Serve the active schema]
        direction TB
        Watch[Watch active version<br/>including rollback] --> Load[Load its archive<br/>Build a schema snapshot]
        Load -->|Unavailable| Pause[Reject new schema requests<br/>Retry without a restart]
        Pause -.-> Watch
        Load -->|Ready and still active| Pin[New requests stick to this snapshot]
        Pin --> Finish[Finish with the same snapshot<br/>even if the active version changes]
    end

    Upgrade -->|Version switch| Serving
    style Serving fill:#eaf3ff,stroke:#4776a8

Fresh installs create initial tables; unchanged schemas skip rebuilding. Code indexing does not block promotion. Rollback reuses retained tables or rebuilds missing ones; both need usable archives.

Related Issues

Relates to #184 (closed). Final serving MR replacing !2452 (closed); follows !2458 (merged) and !2464 (merged), both merged into main.

Testing

After rebasing: 2,585 workspace tests, 183 local integration tests, and 415 server/container tests passed. Clippy, formatting, migration-ledger, and documentation checks passed.

The E2E rerun of 2026-09-11 on the final code passed every upgrade, skip, rollback, and failure-recovery path, including dispatcher-first rollouts, three binary versions serving at once, and a rollback to 0.121.0. The production image and Kubernetes rollout remain untested. Ran both with Astra + Fable.

Performance Analysis

• This merge request does not introduce any performance regression. If a performance regression is expected, explain why.

Agent context

Compatible archives only; retired tables must survive in-flight requests. Metadata read failures preserve the last usable snapshot. Not benchmarked.

Validation report covers commands, scope, and the transient download failure.

The first E2E report recorded the compatibility checks and the failures that drove the final fixes. The 2026-09-11 rerun found none of them; readiness never flapped outside process restarts. The prod and staging gkg_reader role sees every versioned object and gkg_schema_version in system.tables, which the readiness table check needs. Three non-blocking follow-ups came out of it: a deleted catalog key reads back as an empty archive (kv_get ignores delete markers, pre-existing in nats-client; drop the bucket to recover), GetGraphStatus reports the embedded version's indexing progress while serving another archive, and roll-forward after a rollback re-runs the forward migration instead of re-activating. NATS restart requires an indexer restart; catalog-loss recovery requires a dispatcher with the missing archive.