Add siphon-tables image bundling Siphon CDC table definitions

What does this MR do?

Add siphon-tables image bundling Siphon CDC table definitions

We want to publish the CDC table definitions as an independent image that the container generating the CDC configuration can mount as a volume. The image has no runtime and contains only the files Siphon needs to generate that configuration.

This approach keeps Siphon releases isolated from Orbit and GitLab releases. The table definitions depend on the Rails database schema, so they are versioned alongside the Rails code.

Reopen Context

We initially decided to close this MR and build the CDC image in the Rails repository, to stay close to the source of truth and keep things more modular.

As Siphon began rolling out to Dedicated, we ran into two issues that make building the image through CNG the better solution:

  1. Signing: Images built in Rails are not signed with cosign. While signing could be added there, CNG handles it out of the box. This means consumers don't need to trust another key, and we don't have to share a sensitive signing key with additional repositories. (Relates gitlab-org/analytics-section/siphon#342)
  2. Internal releases: Occasionally we ship internal, tag-less releases to deploy high-severity fixes to Dedicated early. Because these releases don't create a tag, no Siphon CDC image is built or tagged in the Rails repo. The simplest way to cover them is to build the image as part of the internal release CNG pipeline. (Relates https://gitlab.com/gitlab-com/gl-infra/gitlab-dedicated/team/-/work_items/14420)

Note: The Siphon release process remains fully decoupled. This change only affects the table definitions, which are still managed in Rails and released as part of the regular CNG/Rails release process.

Relates MR that prepares the Siphon chart to consume this image: feat(chart): support dynamic, externally-manage... (gitlab-org/analytics-section/platform-insights/siphon-helm-charts!63 - merged)

Relates https://gitlab.com/gitlab-com/gl-infra/gitlab-dedicated/team/-/work_items/14420+

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion

Required

  • Merge Request Title, and Description are up to date, accurate, and descriptive
  • MR targeting the appropriate branch
  • MR has a green pipeline on GitLab.com
  • When ready for review, MR is labeled "~workflow::ready for review" per the Distribution MR workflow

Expected (please provide an explanation if not completing)

  • Test plan indicating conditions for success has been posted and passes
  • Documentation created/updated
  • Integration tests added to GitLab QA
  • The impact any change in container size has should be evaluated
  • New dependencies are managed with GitLab forked renovatebot
Edited by Clemens Beck

Merge request reports

Loading
Loading