feat(code-graph-incremental): pipeline runner, artifacts, and graph types

What does this MR do and why?

First slice of a new crate, code-graph-incremental: a code indexer that saves its graph and re-indexes only the files that changed. Nothing is wired to production yet. This MR lands the typed pipeline runner, the time budgets, the canonical node alphabet, and the types a run builds and moves between phases, so each later MR can add one phase in place. The runner is the part every later review depends on, so it goes first.

Relates to #1284 (incremental indexing workstream). Reference implementation: !2454 (closed).

Testing

cargo test -p code-graph-incremental: 7 tests. tests/pipeline.rs covers the public contract: phases chain by type and report every boundary, the run budget stops a run at the next boundary, a skipped item is reported while the run continues, item phases fuse into one pass, the shipped limits load.

cargo clippy -p code-graph-incremental --all-targets -- -D warnings: clean.

Performance Analysis

No runtime change: nothing calls this crate yet.

  • This merge request does not introduce any performance regression. If a performance regression is expected, explain why.
Agent context: long-form analysis, file-by-file walkthroughs, profiler output, alternatives considered

Layout

crates/code-graph-incremental/
  config/limits.yaml            per-file and total budgets in ms
  src/constants.rs              PATH_SEP, WILDCARD
  src/shared/canonical.rs       Canonical: the synthetic node kinds every language rewrites into
  src/shared/sentinel.rs        Limits, Sentinel, Killed
  src/shared/error.rs           LoadError, Error
  src/shared/tags.rs            ReservedTags
  src/pipeline/mod.rs           Pipeline<T>, Phase, ItemPhase + pipe, Context, Observer, Report
  src/pipeline/artifacts.rs     Sources, Changes, ReindexInput, Workset<C>, Listed, Parsed,
                                Rewritten, Canonical, LinkedFile, DirtyGraph, Resolved,
                                Displayed, Exported
  src/pipeline/state.rs         State { trees, edges, resolver, configs }, SourceFile
  src/tree/types.rs             Tag, EdgeKind, Node, Edge, Tree
  src/intern.rs                 Interner, Lang
  src/resolver.rs               Loc, ImportReq, FileIndex, Resolver
  src/env.rs                    Env { limits }
  tests/pipeline.rs

Types marked #[allow(dead_code)] have private fields nobody reads yet; each attribute goes when its impl lands. Env holds only limits here and grows into the language environment (lang, lang_id, rules) in the next MR. The four Cursor-based helpers of canonical.rs arrive with tree/walk.rs.

Phase order

Sources ─Prepare─> Workset<Lazy<SourceFile>>
  ─Each(Parse+Rewrite+Canonicalize+Link)─> Workset<Vec<LinkedFile>>
  ─Insert─> DirtyGraph ─Resolve─> Resolved ─Display─> Displayed
  ─Export─> Exported ─Emit─> Exported
ReindexInput ─Remap─> Workset<Lazy<SourceFile>> ─(as above)

MR stack

MR Contents
1 (this) runner, budgets, errors, canonical alphabet, artifact and graph types
2 Tree impl and walk, Interner, tree-sitter languages, Env, inventory (CodeFilter walk/classify), Prepare + Parse
3 DSL (parser, rules, matching, transform, rewrite), language YAML, Rewrite + Canonicalize
4 linker, SSA, file tree, Link + Insert
5 resolver, Resolve
6 export engine + export.yaml, Display + Export + Emit, index template; YAML suites through the integration-tests-codegraph runner with an engine switch; CI job
7 State save/load, Remap, reindex template, incremental suites
8 orbit index --engine incremental, full index through the existing DuckDB sink
9 snapshot next to the DuckDB file, changed paths from git, incremental CLI
10 default flip per language; old pipeline and its fixtures retire

Exported ids are hashes of path, fqn and byte range (same scheme as code-graph), so a reindex never changes the id of an untouched row; internal tree positions may shift and are never exported.

Why not cherry-pick from !2454 (closed)

The reference branch grew by iteration and its history is not reviewable in slices. Files are copied over at their final shape, one layer per MR, so each MR compiles and tests on its own.

%{all_commits}

Edited by Michael Usachenko

Merge request reports

Loading
Loading