feat(server): query result cache for cursor pagination
What does this MR do and why?
Part of #248 (closed), Depends on !699 (merged) and !698 (merged). Every page of a cursor query currently runs the full server pipeline: ClickHouse query, Rails authorization exchange, redaction, hydration. For an agent paging through 85 results in pages of 20, that's 5 identical round trips to ClickHouse and 5 bidirectional gRPC redaction exchanges with Rails — all producing the same authorized result set, just sliced differently.
This MR adds an in-memory query result cache at the run_query level. The first page runs the full pipeline and caches the complete authorized result (post-redaction, post-hydration, pre-cursor-slicing). Subsequent pages hit the cache and just apply the cursor window — no ClickHouse, no Rails, no redaction.
Page 1: full pipeline → cache → slice [0..20] → return
Page 2: cache hit → slice [20..40] → return
Page 3: cache hit → slice [40..60] → returnThe cache uses moka for lock-free concurrent access with automatic TTL eviction. Cache key is (user_id, hash(query_json without cursor)) so different pages of the same query share an entry.
TTL: 60 seconds — short enough that authorization changes propagate, long enough for multi-page browsing.
Capacity: 16,384 entries with LRU eviction. Cursor slicing moved from OutputStage to run_query so the full result is available for caching before the cursor window is applied.
Memory utilization analysis of LRU cache
Each entry is one full PipelineOutput — up to 1000 rows with hydrated properties. A typical search returning 30 users with 5-10 string columns runs ~2-5 KB.
| Entries | 1 KB/entry | 2 KB/entry | 5 KB/entry | 10 KB/entry |
|---|---|---|---|---|
| 256 | 256 KB | 512 KB | 1.25 MB | 2.5 MB |
| 512 | 512 KB | 1 MB | 2.5 MB | 5 MB |
| 1,024 | 1 MB | 2 MB | 5 MB | 10 MB |
| 2,048 | 2 MB | 4 MB | 10 MB | 20 MB |
| 4,096 | 4 MB | 8 MB | 20 MB | 40 MB |
| 8,192 | 8 MB | 16 MB | 40 MB | 80 MB |
| 16,384 | 16 MB | 32 MB | 80 MB | 160 MB |
At the configured cap with typical entries, we're looking at 30-80 MB. The 160 MB upper bound assumes every cached query returns the maximum 1000 rows with large hydrated payloads — unlikely in practice since most queries use the default limit of 30.