Skip to content

Transparently partition key-value store

Sami Hiltunen requested to merge smh-prefix-db-chk into master

Each storage in Gitaly has a key-value store. The physical key-value store is shared between all partitions in the storage. Logically each partition has their own key-value store. The partitions should not be able to read or write over each other's keys. Currently this is done by just prefixing the keys the partitions are writing manually with a prefix uniquely identifying a given partition. This has so far been sufficient as we've only been writing one key storing the applied LSN. This is going to get more complicated as we're about to introduce support for storing arbitrary key-value data in a partition.

This commit simplifies the situation by making TransactionManager unaware of the keyspace partitioning. Instead, the PartitionManager instantiates each TransactionManager with a key-value store instance that is configured to automatically namespace all key-value operations. This way the transactions and the transaction manager can't accidentally read or write other partition's keys. This fits the architecture also better as the partitions themselves should not be aware of the partitioning as they are lower level than the PartitionManager.

The same mechanism will later be used to namespace client key-value data (data from other Gitaly components) from the internal key-value state maintained by the TransactionManager.

Part of #6088 (closed)

Merge request reports