Support CockroachDB as a backing store
Problem to solve
Reduce the operational burden on teams desiring to deploy a highly available version of Gitlab on-prem or in a cloud/kubernetes environment.
Further details
Regardless of the scale of an installation, no one wants their service to be down. PostgreSQL has been around a long time and is a fine database, but it was designed in a time before the modern concept of highly-available distributed systems. As a result, it is not trivial to run or maintain a PostgreSQL replica let alone scale and shard the data.
CockroachDB1 is a modern RDBMS which implements a PostgreSQL compatible wire protocol2 and is designed to make some of the modern scaling problems easier to manage. While the wire protocol is consistent, the SQL Dialect is not identical. CoackroachDB documents their SQL support3 and has shown a willingness to implement additional features as need arises4.
Proposal
Gitlab should support using CockroachDB in addition to PostgreSQL.
What does success look like, and how can we measure that?
Success can be measured in a number of ways, though many are only tangible to operators of Gitlab.
- High-Availability of the database layer
- Scaling the DB layer in an active-active capacity at a single site becomes feasible.
- Kubernetes-based deployments can leverage StatefulSets to run the database in a manner supported by the database vendor5. charts/gitlab#48
- Simplified geographic replication - CockroachDB supports live replication across a WAN connection. When the database natively supports replication there is less burden on the development teams to handle replication checks and on operators to maintain the replication processes.
There may be measurable performance differences, but the diverse types of queries used in Gitlab makes it difficult to surmise what performance would look like on CockroachDB vs. PostgreSQL.
Possible new features:
- Enable Geographic data constraints - Commercial versions of CockroachDB allows data to remain in specific geographic locales6.
Links / references
-
CockroachDB: https://www.cockroachlabs.com/product/cockroachdb/
↩ -
CockroachDB SQL Layer - https://www.cockroachlabs.com/docs/stable/architecture/sql-layer.html#postgresql-wire-protocol
↩ -
Cocroach SQL Feature Support - https://www.cockroachlabs.com/docs/stable/sql-feature-support.html
↩ -
Cockroach PostgreSQL compatibility - https://www.cockroachlabs.com/blog/why-postgres/
↩ -
CockroachDB and Kubernetes - https://www.cockroachlabs.com/campaigns/kubernetes/
↩ -
Global Clusters - https://www.cockroachlabs.com/solutions/global-clusters/
↩