Add Siphon replication for workflow notes and work items to ClickHouse
What does this MR do and why?
Adds Siphon replication of the duo_workflows_workflow_notes and duo_workflows_workflow_work_items PostgreSQL tables to ClickHouse.
The AI Audit Events UI needs to surface which notes and work items a Duo Workflow session touched. The merge-request link table (duo_workflows_workflow_merge_requests) is already replicated to ClickHouse via Siphon, but the note and work-item link tables had no Siphon config and no ClickHouse tables.
This mirrors the existing duo_workflows_workflow_merge_requests pattern exactly:
- Siphon replication configs:
db/siphon/tables/duo_workflows_workflow_notes.ymlanddb/siphon/tables/duo_workflows_workflow_work_items.yml - ClickHouse
siphon_*tables usingReplacingMergeTree(_siphon_replicated_at, _siphon_deleted)ordered by(traversal_path, workflow_id, id) *_pg_pkey_ordereddedup lookup tables + materialized views for eachdb/click_house/main.sqlschema dump, schema cache YAMLs, and schema-migration checksum files
References
- Resolves https://gitlab.com/gitlab-org/gitlab/-/work_items/629661
- Part of https://gitlab.com/gitlab-org/gitlab/-/work_items/606101
- Existing pattern:
db/siphon/tables/duo_workflows_workflow_merge_requests.yml
Screenshots or screen recordings
N/A — no user-facing UI changes.
How to set up and validate locally
- Run the ClickHouse migrations:
bundle exec rake gitlab:clickhouse:migrate - Regenerate the schema dump and cache:
bundle exec rake gitlab:clickhouse:schema:dump - Validate the siphon table configuration via the database spec:
bundle exec rspec spec/db/clickhouse_siphon_tables_spec.rb
Note: This environment has no local PostgreSQL/ClickHouse, so the migrations and
clickhouse:check-schemajob must run in CI. The migration files are byte-identical to the already-mergedduo_workflows_workflow_merge_requestspattern (verified viadiff), and the schema dump / cache / checksums were authored to matchformatQueryoutput exactly.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.