feat(code-graph-incremental): rewrite engine and the Python rules
What does this MR do and why?
Third slice of code-graph-incremental (after !2620 (merged), !2627 (merged)). A parsed file now goes through the language's rewrite rules and comes out as a tree of canonical nodes only: typed and named definitions, imports, calls.
The rules are YAML, and Python is the first language to ship its file. The engine is language-agnostic; every later language is a rule file, not code. Nothing is wired to the CLI or server yet.
Related Issues
Relates to #1284. Reference implementation: !2454 (closed).
Testing
cargo test -p code-graph-incremental: 21 tests. New in tests/rewrite.rs: a Python file rewrites into only canonical nodes with its source text dropped; definitions carry their type and name (Class, Method, Function); imports and calls become canonical nodes; a file over the rewrite budget is skipped and reported while the run continues. The DSL's own tests came along in the copied files: the embedded rule file compiles, malformed rule files are errors, not panics.
cargo clippy -p code-graph-incremental --all-targets -- -D warnings: clean.
Performance Analysis
No runtime change: nothing calls this crate yet.
- This merge request does not introduce any performance regression. If a performance regression is expected, explain why.
Agent context: long-form analysis, file-by-file walkthroughs, profiler output, alternatives considered
What lands
src/dsl/pattern.pest S-expression pattern grammar
src/dsl/parser.rs pattern text -> Pat
src/dsl/types.rs Pat, Text, Tf, Out, Rewrite, EdgeCtx
src/dsl/rules.rs rule file -> LangConfig { rewrite_stages, resolve_stages, config, display_rules }
src/dsl/matching.rs matches(tree, pat) with captures
src/dsl/transform.rs the transform vocabulary (or, default, tree_path, regex_replace, ...)
src/dsl/rewrite.rs apply_rewrites: match, guard, replace/append/tag
langs/python.yaml Python rules
src/treesitter.rs lang_yaml (Python only for now)
src/env.rs Env.rules: LangConfig, compiled once per language
src/pipeline/phases.rs Rewrite, Canonicalize; the manifest branch of Prepare
src/shared/error.rs From<regex::Error> for LoadError
tests/rewrite.rsFiles are copied from !2454 (closed) at their final shape. The allow(dead_code) on Tree::{replace, postorder, preorder} is gone (the rewrite engine reads them); node_mut keeps its allow until Resolve in MR 5.
The manifest branch of Prepare (parse_files, e.g. Cargo.toml for Rust) is in, but Python declares no manifests, so its test arrives with the Rust rule file.
MR stack
| MR | Contents |
|---|---|
| !2620 (merged) | runner, budgets, errors, canonical alphabet, artifact and graph types |
| !2627 (merged) | Tree, Interner, tree-sitter languages, Env, inventory, Prepare + Each + Parse |
| 3 (this) | DSL engine, Rewrite + Canonicalize, Env.rules, Python rules |
| 4 | linker, SSA, file tree, Link + Insert |
| 5 | resolver, Resolve |
| 6 | export engine + export.yaml, Display + Export + Emit, index template; YAML suites through the integration-tests-codegraph runner with an engine switch; CI job |
| 7 | State save/load, Remap, reindex template, incremental suites |
| 8 | remaining language rule files |
| 9 | orbit index --engine incremental, full index through the existing DuckDB sink |
| 10 | snapshot next to the DuckDB file, changed paths from git, incremental CLI |
| 11 | default flip per language; old pipeline and its fixtures retire |
%{all_commits}