Replicate ai_governance_sessions to ClickHouse with Siphon

What does this MR do and why?

Replicates the new ai_governance_sessions Postgres table to ClickHouse using Siphon, the same approach used for duo_workflows_workflows. The AI Governance dashboard reads from ClickHouse on GitLab.com, with Postgres only as a fallback, so without this replication, rows in the new table would not be visible there.

Notes for reviewers
  • Files were generated with rails generate gitlab:click_house:siphon ai_governance_sessions --with-traversal-path, then hand-edited:
    • Changed the ClickHouse sort key to (traversal_path, session_started_at, id) instead of the generator's created_at, so ClickHouse orders sessions the same way as the Postgres index (namespace_id, session_started_at, id) and the dashboard's Start time column.
    • Added session_started_at to the lookup table siphon_ai_governance_sessions_pg_pkey_ordered and its materialized view, since the lookup table needs every sort-key column so Siphon can find and replace old rows on update.
  • The reconcile config only lists namespace_id, because that is the only sharding key defined in db/docs/ai_governance_sessions.yml.

References

Screenshots or screen recordings

Not applicable, this MR has no UI change.

How to set up and validate locally

  1. Run bundle exec rspec spec/db/clickhouse_siphon_tables_spec.rb (80 examples, 0 failures locally).
  2. Optionally, with Siphon enabled in GDK, run gdk reconfigure && gdk restart siphon, then in a rails console run ClickHouse::Client.select('SELECT count() FROM siphon_ai_governance_sessions FINAL', :main).

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Hitesh Raghuvanshi

Merge request reports

Loading
Loading