v0.7.0 — P1: bro hallucinates from LLM memory instead of trajectory DB — undermines TMB's design premise
## Why this is P1 TMB is built on a load-bearing premise: **the trajectory DB + file_registry replace the LLM's memory as the source of truth for project facts.** That is the whole reason for the project's existence — without it, TMB is just another agent harness paying full LLM-memory tax. A live-fire failure on 2026-05-17 proved the premise is not enforced: Bro filed issue #2904 ("Slim bro's always-on skill set") with these false claims: - Listed `tmb_default_repo` as a skill — **does not exist** in `plugin/skills/` - Listed `tmb_scan` as a skill — actually `commands/scan.md`, a slash command - Claimed `tmb_planning` is "~180 lines" — actually **292 lines** - Claimed `tmb_review` is "~200 lines" — actually **274 lines** - Listed 6 skills — there are actually **8** in `plugin/skills/` - Implicit reference to `agents/bro.md` — **does not exist**; bro is the main CC session, persona via `CLAUDE.md` + skills Zero of those claims were checked against the trajectory DB or the live file tree. They were confabulated from prior-session LLM memory. The Human caught it on review. If this had shipped to a fresh user reading #2904 to understand TMB's load profile, every fact they relied on would be wrong. ## Why CLAUDE.md's current advice does not prevent it `CLAUDE.md` ("You are bro") includes a "Before answering — verify context" table: | Situation | Where to look | |---|---| | Git clean | trajectory DB's `file_registry` | | Git dirty | diff vs `file_registry`; Read / Glob / Grep only changed files | | ... | ... | This is **advisory prose**. There is no hook, no MCP gate, no tool-side check that *enforces* "you must consult file_registry before naming a file or symbol in a write." The advice is loaded into context every session and ignored under cognitive load (e.g., during a 15-issue burst where the LLM falls into the cheapest path: recall). Per the user-memory rule `feedback_no_negative_prompt_rules.md`, adding "never confabulate" or "always verify" to bro's prompt is pink-elephant doctrine and structurally ineffective. The fix has to be **structural** — a hook, a server-side check, or a tool that makes the right thing the easy thing. ## Root-cause hypotheses (to triage, not decide) 1. **No enforcement gate on issue/discussion writes.** `issue_create`, `discussion_append`, `audit_log` all accept arbitrary markdown bodies. Nothing scans for file paths or symbol names and verifies them. 2. **`file_registry` granularity is too coarse to be useful for facts about a file.** Summaries are 1–2 line role descriptions; they don't carry line counts, exported symbols, dependency edges, or recent changes. Even if bro consulted `file_registry`, he couldn't answer "how long is tmb_planning?" 3. **No on-write reverse-index check.** When bro writes "tmb_default_repo" in an issue body, nothing greps the live tree to confirm a `tmb_default_repo` skill exists. 4. **Skill descriptions don't gate.** `tmb_planning` description says "Self-contained — everything bro needs is here" — encouraging bro to act from skill body rather than re-query DB/source. 5. **The 5-minute prompt cache makes recall cheaper than verification.** A Read costs tokens + latency; LLM recall is free + instant. Without a structural cost, the LLM will choose recall every time. ## Plan (structural, per `feedback_no_negative_prompt_rules.md`) ### Phase 1 — claim-verification hook (highest leverage) Add a PreToolUse hook on `mcp__.*trajectory-server__issue_(create|update_description)` and on `mcp__.*trajectory-server__discussion_append`: 1. Extract candidate file paths from the body using regex: `\b(skills|agents|commands|scripts|mcp|hooks|docs|templates)/[a-zA-Z0-9_/-]+\.(md|ts|js|sql|sh|json)\b` 2. Extract candidate skill names: `\btmb_[a-z][a-z0-9-]*\b` 3. For each, check existence: - File paths: `[ -e "$CLAUDE_PLUGIN_ROOT/$path" ]` - Skill names: `[ -d "$CLAUDE_PLUGIN_ROOT/skills/$name" ]` 4. If any unverified claims found: **block the write** with `{"reason": "unverified_claims", "candidates": [...], "remedy": "Read the file or grep the tree; if the symbol does not exist, remove the claim or correct it."}` 5. Bypass via explicit `__TMB_CLAIMS_VERIFIED=true` env or a structured tool param like `evidence_paths: [...]` — forces bro to either prove the claim is real or acknowledge it's speculative. ### Phase 2 — enrich `file_registry` for verifiability Add structured columns the LLM can query cheaply instead of re-reading: - `line_count INTEGER` — kept in sync by `scan_run` - `exported_symbols TEXT` — JSON array, populated by Phase-2 summarizer for TS/JS/Python files - `kind TEXT CHECK(kind IN ('skill','agent','command','hook','source','test','doc','config'))` — derived from path - `frontmatter_json TEXT` — for `.md` files with `---` blocks (skills/agents/commands) Make these queryable via `file_registry_list(kind='skill', fields=['path','line_count','frontmatter_json'])` — bro can answer "list all skills with line counts" in one tool call, no re-discovery, no recall. ### Phase 3 — bro skill update (small, positive) Update `skills/tmb_planning/SKILL.md` to add a "Claim verification" step: > Before writing factual claims into issues/discussions/audit (file paths, symbol names, line counts, structural assertions), call `file_registry_list` with the relevant `kind` filter. If a claim isn't backed by a registry row or a fresh Read, mark it explicitly as inference: "based on prior-session impression" or "estimated". This is *positive* doctrine (do this) not negative (don't confabulate), and it's backed by Phase 1's structural enforcement so the LLM can't ignore it without the write failing. ### Phase 4 — measurement Add an `audit` event `claim_verification_blocked` emitted by the Phase-1 hook. Track frequency over a release cycle. Target: trend approaches zero as bro internalizes the new flow. ## Acceptance criteria - New PreToolUse hook `scripts/hooks/verify-write-claims.sh` blocks `issue_create`/`issue_update_description`/`discussion_append` when their body references a file path or skill name that does not exist in the live tree. - New L4 simulation: attempt to call `issue_create` with a body referencing `tmb_default_repo` (non-existent) — verify the hook blocks and the error message is actionable. - New L4 simulation: attempt to call `issue_create` with a body referencing `skills/tmb_planning/SKILL.md` (real) — verify it passes. - `file_registry.line_count` + `kind` columns ship in v3 schema migration. - `scan_run` populates the new columns on every fresh scan. - `tmb_planning` SKILL.md updated with the Claim-verification step. - L5 dogfood: a representative 5-turn session shows zero `claim_verification_blocked` audit events. ## Out of scope (deferred to v0.7+) - Semantic claim verification ("Is 'TMB is built on…' accurate?") — that needs an evaluator agent, not a regex hook. - Verifying claims in PR descriptions / commit messages / external docs — start with DB-write surface. - Auto-correction (rewriting "tmb_default_repo" to a similar real skill) — explicit Human review preferred over silent rewrite. ## Coordination - Pairs with #2916 (compact responses) — both want `file_registry` to be more queryable. - Pairs with #2906 (cap field sizes) — both want v2→v3 schema migration. - Pairs with #2918 (cache discipline) — verification cost should be cache-friendly so it doesn't itself become a token burn. - Directly addresses the failure mode that produced the wrong description in #2904. ## Why P1 This is not a feature gap; it's an architecture-integrity issue. TMB's value proposition collapses if bro can confabulate facts into the DB without challenge. Every other token-burn or quality issue is downstream of this: even after we make skills slimmer (#2904) or paginate reads (#2905), bro will still hallucinate at the next opportunity if the structural prevention isn't in place.
issue