feat(indexer): add NATS entity versioning for blue-green deployments

What does this MR do and why?

Introduces a version identifier into all GKG-owned NATS entity names so two versions can run side by side during blue-green deployments without competing for messages, locks, or indexing progress.

Versioned entities: JetStream streams (GKG_INDEXER_V{N}, GKG_DEAD_LETTERS_V{N}), KV buckets (indexing_locks_v{N}, orbit_indexing_progress_v{N}), subjects (v{N}.<base>), and durable consumer names. The version is derived from the graph schema version. A cleanup function deletes all NATS entities for a retired version.

NatsVersioner.resolve_stream_and_subject() is the single resolution point used by both the broker and dead letter modules.

Relates to #853 (closed)

Testing

Unit tests cover versioner formatting, managed stream/bucket table sync, resolve_stream_and_subject for managed and unmanaged subscriptions, and dead letter subject construction through the versioner.

Validated locally against a running NATS server (GDK) using nats CLI queries. Confirmed all GKG-owned entities are versioned (GKG_INDEXER_V62, GKG_DEAD_LETTERS_V62, indexing_locks_v62, orbit_indexing_progress_v62), subjects carry the v62. prefix, subscription consumers include the version tag, and Siphon streams remain unversioned.

Performance Analysis

  • This merge request does not introduce any performance regression. If a performance regression is expected, explain why.
Agent context — long-form analysis, file-by-file walkthroughs, profiler output, alternatives considered

File-by-file walkthrough

  • crates/indexer/src/nats/versioning.rs (new) — NatsVersioner struct with formatting methods (stream, bucket, subject, tag, resolve_stream_and_subject). Global NATS_VERSIONER initialized from SCHEMA_VERSION. MANAGED_STREAMS/MANAGED_BUCKETS tables for cleanup discovery. cleanup_version() deletes all NATS entities for a retired version. Tests verify formatting, managed table sync with source constants, and resolution for managed vs unmanaged subscriptions.

  • crates/indexer/src/engine/dead_letter.rs — Removed resolved_stream parameter from dead_letter_subject, dead_letter_subscription, and DeadLetterEnvelope::new. These now resolve stream names internally via NATS_VERSIONER.resolve_stream_and_subject() and apply the version prefix to subjects. Tests updated to use SCHEMA_VERSION instead of hardcoded version strings.

  • crates/indexer/src/nats/broker.rs — Replaced local resolve_names function with NATS_VERSIONER.resolve_stream_and_subject(). publish_dead_letter simplified: no longer pre-resolves stream or wraps subjects — dead letter module handles it. ensure_managed_streams and ensure_dead_letter_stream use versioner for stream names and subject filters. consume_pending injects version tag into dispatch consumer names for unmanaged (Siphon) streams.

  • crates/indexer/src/schema/completion.rs — Hooks NATS entity cleanup into the existing migration completion path via cleanup_version.

  • crates/gkg-server-config/src/nats.rs — Added consumer_inactive_threshold config for Siphon consumer auto-deletion.

  • config/default.yaml — Default consumer_inactive_threshold_secs: 3600.

  • crates/indexer/src/indexing_status.rs, crates/indexer/src/lib.rs — KV bucket creation routed through versioner.

  • Integration tests — Updated to use versioned entity names in assertions and setup.

KV bucket versioning: single boundary

NatsBroker is the single versioning boundary for KV operations, as it already is for streams and subjects. It implements KvServices and auto-versions in its KV methods (kv_get, kv_put, etc.) via NATS_VERSIONER.bucket() — callers pass raw bucket constants (e.g. INDEXING_LOCKS_BUCKET) and never version by hand. IndexingStatusStore is constructed with the broker (IndexingStatusStore::new(broker)) and routes its progress-bucket reads/writes through it, so versioning happens in one place rather than relying on convention at each call site.

%{all_commits}

Edited by Aaron Algutifan

Merge request reports

Loading
Loading