## Summary Follow-up from a Slack discussion (2026-10-05/06) about how Siphon gets its CDC table definitions, written up so the options can be proposed and compared here. Requested by @arun.sori. @WarheadsSE asked to move the refactor discussion into an issue. The near-term GDG blocker (no crane, no runtime pulls from `registry.gitlab.com`, signed artifact) is tracked in https://gitlab.com/gitlab-org/analytics-section/siphon/-/work_items/331. This issue covers the broader design question: **how should table definitions get from GitLab Rails to Siphon in a way that works for air-gapped / FIPS installs (GDG and Self-Managed) without tightly coupling Siphon and GitLab releases?** ## Background - Today the table definitions are a tar/OCI artifact (`gitlab-siphon-tables:v<gitlab_version>-ee`). Monolith CI builds it and tags it per GitLab release, then pushes it to `registry.gitlab.com`. Siphon always gets the exact schema of the running GitLab version. Original design discussion: https://gitlab.com/gitlab-org/cloud-native/charts/dip-chart/-/work_items/7#note_3414808432 - On Dedicated, an init container (`crane`) pulls and extracts the artifact into a shared volume, because the ImageVolume feature gate is off on EKS. - Air-gapped / GDG problems: no access to `gitlab.com` / `registry.gitlab.com`, the artifact is unsigned, and copying, validating and extracting it needs extra tooling plus pod identity for ECR/KMS. Solving this cleanly for Self-Managed also solves it for GDG. - @ahegyi's constraints: the ClickHouse schema is tied to the PG schema. On migrate, **CH migrations run before PG migrations**, so replication doesn't lose data (for example, a new column is written immediately). CH must also be migrated before the new app version runs, because the app depends on the new schema. The table config itself only needs the new table YAMLs to be brought in after migrations, followed by a Siphon restart. - @WarheadsSE: we don't want tight Rails/Siphon coupling, but there is a _real_ tie between them because of the database schema state. ## Options discussed | # | Option | Notes / concerns raised | |---|---|---| | 1 | **Hardcode / embed table definitions in Siphon** (binary, image or Helm chart). POC: https://gitlab.com/gitlab-org/analytics-section/siphon/-/merge_requests/605 | Couples Siphon releases to GitLab releases. Needs a Siphon x GitLab compatibility matrix and its own release/backport policy (e.g. how many GitLab versions each Siphon version supports). Delivery and Siphon team members consider this unmaintainable. @arun.sori doesn't expect the POC to go in as-is. | | 2 | **Signed tables image + runtime verify/pull/unpack** (Delivery proposal). Add an init container ahead of `generate-cdc-config` that (1) verifies the signature with cosign, (2) pulls and unpacks with crane, and (3) shares the result via a volume. This emulates an OCI volume mount. Fall back to it only when the definitions aren't mounted externally. Make the image path overridable so air-gapped installs can point to a private mirror. | Keeps Siphon and GitLab decoupled. Open questions: **(a)** where and how to sign the artifact: integrate cosign into the Rails pipeline, or move the image to CNG? **(b)** which image ships cosign and crane? @arun.sori suggests the Siphon image. GDG notes: crane still isn't FIPS-approved, and a registry ref passed as an argument isn't rewritten by Kyverno (see https://gitlab.com/gitlab-org/analytics-section/siphon/-/work_items/331). It would also need pod roles/identities for ECR + KMS. | | 3 | **Runnable tables image as an init container** (option A in https://gitlab.com/gitlab-org/analytics-section/siphon/-/work_items/331). Publish a signed, minimal image that copies the tables into an `emptyDir`. The chart renders it as a normal `image:` field (e.g. implement `split.tablesVolumeMode: initCopy`). | Gets mirrored and rewritten like any other image, with no crane at runtime. One image per GitLab version. Needs a FIPS-acceptable base. This is the option the GDG side prefers. | | 4 | **Siphon fetches config from the tenant's Rails API** (@michaelangeloio). On startup, check e.g. `api/v4/internal/dip/version`, then fetch table definitions / SQL from e.g. `api/v4/internal/dip/tables` and reconcile periodically. | The tenant's own Rails already has the schema knowledge, so no external access is needed. Concerns: how the API would know migrations have completed; Rails already has a migration framework, and CH migrations shouldn't be driven through an API; it needs a new endpoint, auth and a startup dependency; and backports would be needed for older GitLab versions. | | 5 | **Decouple CH migration timing / multi-version schema** (exploratory). Ideas: pause NATS consumer inserts until migrated, serve two schemas at once and switch traffic (like Orbit), or put a query API in front so Rails never talks to CH directly. | CH still has to migrate before the new app version runs. CH multi-version compatibility wasn't a requirement originally; the design followed PG. We'd need to revisit the PG migration tooling and every migration step type before we know if this is feasible. Longer-term. | | 6 | **Keep ImageVolume** and wait for EKS to enable it by default | Version-gated and doesn't unblock GDG in the near term (see https://gitlab.com/gitlab-org/analytics-section/siphon/-/work_items/331). | Related: ADR discussing similar Rails <-> modular service coupling pain points: https://gitlab.com/gitlab-com/content-sites/handbook/-/merge_requests/20596 ## Open questions / next steps - [x] Delivery to set expectations on the Rails <-> Siphon version-compatibility policy. This also sets the policy for Self-Managed. - [x] Decide where the tables artifact is built and signed (Rails pipeline with cosign vs CNG). - [x] Decide the runtime delivery mechanism (option 2 vs 3) and where any tooling lives. - [x] Short sync (~15 min) to align on the asks (requested by @WarheadsSE). - [ ] Longer-term: assess whether CH multi-version schema compatibility / delayed migration is feasible (option 5). /cc @arun.sori @michaelangeloio @ahegyi @WarheadsSE