fix(ontology): declare missing edge variants and enforce them at load time
What does this MR do and why?
Three edge types shipped writing graph rows that no query can traverse, because the edge never declared the endpoint pair its ETL produces: dependency-to-project (787k orphaned rows in prod), job auto-cancellation (26.4M rows pointing at the wrong node type), and the new package-file-to-project link. This declares the missing pairs and makes ontology loading fail whenever ETL and declarations disagree, so the gap cannot ship again. It also decodes package-file checksums, which were stored double-encoded and unfilterable, and removes the last partition-era projections from the node tables.
Related Issues
Closes #962 (closed)
Testing
cargo nextest run --workspace unit suite (2123, includes five new loader-validation tests, mutation-checked, plus the two FK-planner tests updated off the corrected variant), and the package scenario suite under Docker (8 scenarios, including end-to-end checksum-decode and null-pipeline assertions). mise ontology:validate and the query-language docs gate pass. The rendered PackageFile extract was executed verbatim against production ClickHouse and returns correctly decoded digests.
Performance Analysis
Additive. The decode runs once per extracted row at index time; no storage-layout changes.
- This merge request does not introduce any performance regression.
Agent context — long-form analysis, file-by-file walkthroughs, profiler output, alternatives considered
Variant gap
Ontology.edges is populated only from edge YAML (loading/mod.rs); node etl.edges never synthesize variants, while the indexer materializes rows from them regardless (modules/sdlc/plan/input.rs). Undeclared pairs are invisible to planning, FK-lowering, scope resolution, and get_graph_schema introspection. Prod evidence (ClickHouse Cloud, gkg database):
v77_gl_edge,relationship_kind='IN_PROJECT',source_kind='Dependency': 787,226 rows; introspection showsDependency: out[0]whiledata_model.mddocuments the traversal.v77_gl_ci_edge,relationship_kind='AUTO_CANCELED_BY':Job → Pipeline26,366,201 rows vs declared variantJob → Job. Rails:CommitStatus belongs_to :auto_canceled_by, class_name: 'Ci::Pipeline'— the ETL was right, the variant was wrong, and FK-lowering joined against the wrong table.in_project.yamlgainsPackageFile → Project(shipped in !1960 (merged) without it) andDependency → Project;auto_canceled_by.yaml's job variant now targetsPipeline.
validate_etl_edges_match_variants (loading/mod.rs) walks every node ETL edge mapping (including to_column type-mapped targets, with direction-aware pair flipping) and every join-table edge ETL config, failing load on any undeclared pair. Five tests exercise the full load_with path via in-memory YAML; disabling the call makes exactly the four rejection tests fail. Two compiler FK-planner tests (fk_star_unfiltered_join_narrow_uses_candidate_scan, fk_center_traversal_keeps_narrow_scan) asserted planning against the buggy Job → Job variant; they now exercise the same self-referential FK shape via Pipeline → Pipeline.
file_sha256 decode
PG stores the digest as bytea holding the ASCII hex string; Siphon replicates the PG hex-escape form, so 32.46M prod rows carry 130-char \x-prefixed blobs (5 stray malformed rows, remainder NULL). The node ETL switches to query ETL (precedent: Finding, MergeRequestDiffFile) with if(match(file_sha256, '^\\x[0-9a-fA-F]{128}$'), unhex(substring(file_sha256, 3)), NULL) — verified against prod rows to yield proper 64-char digests; malformed values become NULL. The extract also has to project package_id explicitly now (table ETL auto-appended FK columns; query ETL selects are authoritative) — caught by the scenario suite, which now seeds a prod-shaped encoded value and asserts the decoded digest, plus a malformed value asserting NULL. SCHEMA_VERSION 79 → 80.
Alternatives considered
Auto-synthesizing variants from node ETL mappings — rejected: variants carry author-declared scope and FK metadata; validation keeps declaration explicit while making drift impossible. Dropping file_sha256 instead of decoding — rejected: the checksum is the point of per-artifact provenance. Lightweight _part_offset projections on gl_package_file (mirroring gl_dependency) — included initially, dropped in review; gl_dependency's own projections are removed in the same pass, so no node table carries projections and the removal rides the same v80 migration.