feat(ontology): add statistics schema config for cost-based query planning

What does this MR do and why?

Part of #812 (closed). Adds a statistics section to schema.yaml that declares automated column statistics collection via AggregatingMergeTree MVs. Three stat types, auto-derived from ontology property types:

Property type Stat table Purpose
boolean / enum / selectivity: low gkg_column_stats Per-value frequency (categorical)
string gkg_token_stats Token frequency (text)
int64 / timestamp / date / float gkg_histogram_stats Equi-depth histogram (continuous)
uuid / virtual / non-filterable skipped

Stats are partitioned by traversal_path for namespace-level granularity. Global entities (User, Runner) use an empty partition key. Total row count per entity comes from the histogram table's id column.

This is schema config + ontology types only. DDL generation (tables, MVs, dictionary) and compiler integration follow in subsequent MRs.

First step toward cost-based join ordering in the query compiler.

Testing

Ontology validation passes. Ontology crate builds clean. No runtime behavior change -- config is loaded but not yet consumed by DDL generation.

Performance Analysis

Schema config only, no query behavior change.

  • This merge request does not introduce any performance regression. If a performance regression is expected, explain why.
Agent context

The statistics config enables a future query planner to estimate cardinality and filter selectivity using real data instead of heuristic annotations. The planner will use these stats to:

  1. Choose optimal join order (start from the most selective table)
  2. Decide whether narrowing CTEs are worth the cost
  3. Estimate intermediate result sizes for multi-hop queries

Three stat types cover all filter patterns in the DSL:

  • Value frequencies for equality/IN on categorical columns (state = 'merged')
  • Token frequencies for contains/starts_with/ends_with on text columns (path LIKE '%.rb')
  • Histograms for range filters on continuous columns (created_at > '2024-01-01')

Auto-derivation rules in stats_columns_for() categorize all 384 filterable properties across 28 entities without manual listing. The schema config is 15 lines instead of the 130 it would take to list every column manually.

Global entities (User, Runner) have no traversal_path so they get global stats with an empty partition key. stats_partition_key_for() returns None for these, signaling the DDL generator to omit the partition column from the MV GROUP BY.

%{all_commits}

[skip schema-version-check]

Edited by Michael Usachenko

Merge request reports

Loading
Loading