Skip to content

Cloud Connector for Optimize group analytics

Update – February 2025:

We are unsure if Cloud Connector is the right approach for ClickHouse on Self-Managed. As the team continue the effort to support GitLab Dedicated with ClickHouse Cloud, we plan to Mature the BYO ClickHouse integration.

Problem

The new AI Impact and high-performance Contribution Analytics are dependent on ClickHouse.

For .com customers ClickHouse is available seamless, but self-manage customers need to bring their own CH for these features.

(This it's not related to Dedicated anymore - in progress as detailed in this issue)

There are several problems with BYO CH for SM:

  1. The implementation and maintenance is complex, resulting high TCO for customers.
  2. It extends the adoption time and decreases Gitlab's competitiveness.
  3. Its create a fragmented architecture that lacks a unified approach, making it difficult to support, adding risks for breaking change and adds barriers for innovation.

Proposal

Using Cloud Connector to allow Self-Managed instances to query data from .com ClickHouse. The end goal is to provide a consolidated solution for all analytical solutions within Gitlab, not specific for groupoptimize.

  1. Using CH Cloud services adding a new CH instance for self-managed instances.
  2. Adding a new module gateway - cloud.gitlab.com/analytics ( or clickhouse.cloud.gitlab.com), under cloud.gitlab.com.
  3. All self-managed GitLab instances will use the same hosted CH instance on clickhouse.cloud.gitlab.com.
  4. For the MVC start only with the data needed for AI Impact.
  5. For the MVC start with only 1 regional.

Related info

Edited by Haim Snir