fix(ontology): compute full_path from namespace hierarchy

Summary

Part of #388 (closed).

Replaces the complex traversal_ids self-join subquery for full_path computation with a simple LEFT JOIN siphon_routes. The routes table is the authoritative source for full_path in Rails and is now replicated via siphon CDC.

This approach was recommended by @ahegyi on the previous iteration of this MR.

Changes:

  • group.yaml / project.yaml -- LEFT JOIN (SELECT * FROM siphon_routes FINAL) instead of traversal_ids subquery
  • fixtures/siphon.sql -- added siphon_routes table DDL
  • Integration tests for rename, parent rename cascade, project move, and no-route fallback

Depends on: gitlab-org/gitlab!229755 (merged) (siphon config + ClickHouse migration for routes table)

How to test locally

# Run integration tests (requires Docker for testcontainers)
mise test:integration

# Or run just the SDLC namespace tests
cargo nextest run --test containers -E 'test(sdlc::namespace_indexing)' --all-features
E2E validation with GDK VM workspace

Validated using gkg-gdk-vm-workspace.

Integration tests (30 subtests, all passing):

  • computes_full_path_for_top_level_group -- single segment path from route
  • computes_full_path_for_nested_subgroups -- 3-level deep paths from routes
  • computes_full_path_for_projects -- project full_path from routes
  • route_rename_updates_full_path -- group rename via route update (checkpoint + incremental reprocessing)
  • child_route_reflects_parent_rename -- parent rename cascades to child groups via updated routes
  • no_route_falls_back_to_slug -- graceful fallback to slug when no route exists
  • project_route_update_changes_full_path -- project move between groups updates full_path

Live GDK VM E2E tests (siphon CDC pipeline):

  • Created 4-level deep group hierarchy via GitLab API: top-org/platform-team/backend/api
  • Created projects at levels 1, 3, and 4
  • Siphon replicated all routes to ClickHouse within seconds
  • Mid-level group rename (platform-team -> engineering): all child routes cascaded correctly through siphon CDC
  • Top-level group rename (top-org -> mega-corp): all 4 group levels + 3 projects cascaded within 30s
  • Project transfer (project from toolbox to gitlab-org): route updated to gitlab-org/gitlab-smoke-tests
  • GKG ETL query verified correct full_path at every nesting level

ClickHouse implementation details:

  • FINAL is required on routes subquery for ReplacingMergeTree deduplication
  • ClickHouse LEFT JOIN on non-nullable String returns empty string (not NULL), so coalesce does not work. Used if(route.id > 0, route.path, namespace.path) instead.
Edited by Michael Angelo Rivera

Merge request reports

Loading
Loading