feat(code-graph-incremental): export the graph as ontology tables
What does this MR do and why?
An incremental code indexing run now ends with a set of RecordBatch that correspond to the ontology's local tables: gl_file, gl_directory, gl_definition, gl_imported_symbol, gl_edge, with the same columns and the same content-hashed ids as production writes. Nothing is wired to the CLI or server yet.
Related Issues
Relates to #1284. Reference implementation: !2454 (closed).
Testing
cargo test -p code-graph-incremental: 45 tests. New in tests/export.rs: every file becomes a row with its size and reason, and src becomes a directory row; definitions, imports and edges export with the expected fqn, type, file and line; two runs over the same repository export identical ids in every table; Emit hands every table to the sink and keeps the graph.
cargo clippy -p code-graph-incremental --all-targets -- -D warnings: clean.
Performance Analysis
No runtime change: nothing calls this crate yet.
- This merge request does not introduce any performance regression. If a performance regression is expected, explain why.
Agent context: long-form analysis, file-by-file walkthroughs, profiler output, alternatives considered
What lands
config/export.yaml entities (source kind, id seed and columns, column sources), containment and graph edges
src/export/parse.rs export.yaml -> compiled plan, validated against the ontology
src/export/mod.rs export(state, lang, ontology, envelope) -> Vec<(table, RecordBatch)>; Envelope, Scalar
src/pipeline/phases.rs Display (display rules -> tags), Export, Emit
src/templates.rs index(): Prepare, Each(Parse+Rewrite+Canonicalize+Link), Insert, Resolve
src/shared/error.rs Error::Export(ArrowError)
tests/export.rsFiles are copied from !2454 (closed) at their final shape; templates::reindex waits for Remap and snapshots in MR 7.
Ids
export.yaml declares each entity's id as a hash over [project_id, branch] plus a seed and named columns: file over path; def over file_path, fqn, span; import over file_path, import_path, identifier_name, span. This is the scheme code-graph uses, so the two engines produce the same id for the same row, and a reindex never changes the id of an untouched row. The test ids_depend_on_content_not_on_the_run pins it.
MR stack
| MR | Contents |
|---|---|
| !2620 (merged) | runner, budgets, errors, canonical alphabet, artifact and graph types |
| !2627 (merged) | Tree, Interner, tree-sitter languages, Env, inventory, Prepare + Each + Parse |
| !2629 (merged) | DSL engine, Rewrite + Canonicalize, Env.rules, Python rules |
| !2631 (merged) | SSA, linker, Link + Insert |
| !2633 (merged) | resolver, project tree, Resolve |
| 6 (this) | export engine, Display + Export + Emit, index template |
| 7 | YAML suites through the integration-tests-codegraph runner with an engine switch; CI job |
| 8 | State save/load, Remap, reindex template, incremental suites |
| 9 | remaining language rule files |
| 10 | orbit index --engine incremental, full index through the existing DuckDB sink |
| 11 | snapshot next to the DuckDB file, changed paths from git, incremental CLI |
| 12 | default flip per language; old pipeline and its fixtures retire |
%{all_commits}