Skip to content

Implement basic transaction management with write-ahead logging

Sami Hiltunen requested to merge smh-log-worker into master

Gitaly is currently lacking in transaction control. Each write coming in launches their own Git commands which operate on the repository concurrently. This makes transaction management difficult. It's difficult to optimize the writes as they are being done from multiple locations without synchronization. The concurrent writers may step on each others toes and surface lock conflicts to the users. Recovering from crashes is also difficult as Gitaly is not logging the modifications it is about to perform and thus loses the transaction state on crashes. There's also no clear notion of ordering which further complicates replication related matters. It's not easy to say which writes a repository is missing and which not.

We've recently designed a new replication architecture for Gitaly. The new architecture relies on a replicated write-ahead log. The write-ahead log defines a clear order of writes and aids in crash recovery. A single writer will be operating on a repository which makes further optimizations such as write batching easier.

This commit implements the first steps towards the new architecture by implementing the TransactionManager. The TransactionManager will be responsible for transaction management of a single repository. It will be the single goroutine that writes into a repository and is invoked by all other locations in the code that wish to write. It will also be responsible for synchronizing reads by ensuring they see the changes they are supposed to see.

TransactionManager implementation introduced here does not contain the full implementation but aims to provide a basis for future iteration. For now, it implements basic write processing with a write-ahead log. It processes writes one-by-one by verifying references, logging the changes and finally applying the changes to the repository. It also supports recovering from the write-ahead log should the log processing be interrupted. The reference verification behavior can be tuned on a per transaction level to match behavior Git's --atomic or --force push flags.

The TransactionManager stores the state related to the write-ahead log in BadgerDB, which is a key-value store that will be local to each Gitaly storage. The values are marshaled protocol buffer messages.

This iteration is mostly concerned with the reference updating logic. Pack files are not handled yet as aren't the internal references they need. Symbolic references, namely for updating the default branch, are not handled yet either. The writes are processed one by one and are acknowledged after applying them to the repository. Given that, there's not separate logic needed for read synchronization yet either.

The goal here is to set the initial interface and log processing, and to lock down the reference updating logic with tests so we can later on safely start iterating on the internals of the TransactionManager and start adding support for the missing functionality.

Part of #4514 (closed)

Edited by Sami Hiltunen

Merge request reports