Skip to content

Make storages fully independent in PartitionManager

Sami Hiltunen requested to merge smh-database-per-storage into master

Gitaly's PartitionManager is responsible for managing partitions. This entails spawning TransactionManagers as needed to process transactions. Currently all of the partitions share the same database and staging directory. This is not correct as the storages generally reside on different disks.

Doing copy free moves of files and hard links across disks is not possible. TransactionManager relies on this functionality to log pack files without copying them.

Each storage should have their own embedded database. The storages are meant to be fully independent of each other and should keep functioning even if one of them fails or is removed, so they need to contain all of the data they need to function. If they share the database, all storages will lose their state if the disk hosting the storage holding the shared database fails. There's also a performance benefit for having a database separately for each storage, as the database performance on a single storage doesn't affect other storages if all of them have their own.

It's also not necessary to synchronize reads or writes between storages. If the transactions target different storages, they are clearly working on separate pieces of data.

This commit addresses the above points and makes the storages fully independent of each other by:

  • Giving each storage their own staging directory.
  • Giving each storage their own database.
  • Using separate locks for accessing data on different storages.

Together, these reduce unnecessary contention transactions to differnet storages. As each storage is self contains all state, they become independent failure domains. They can also be moved between nodes freely as they don't depend on data in other storages.

Closes #5125 (closed)

Merge request reports