Skip to content

Enable snapshot reads of hooks

Sami Hiltunen requested to merge smh-hook-index into master

When a transaction begins, its read snapshot is determined. The snapshot is a consistent view of the data at a given point in time. Every operation through out the transaction should read this snapshot regardless of other writes happening and committing concurrently. To do so without blocking concurrent modifications, databases generally use multiversion concurrency control which is a technique where in multiple versions of a given data point is retained until they are no longer needed by open transactions.

Gitaly's TransactionManager is currently storing multiple versions of custom hooks on the disk to enable snapshot reads of them. However, there's currently no way to know which hooks are included in given transaction's snapshot. This commit addresses that issue by including the right version of the hooks in the transaction's snapshot.

When a transaction begins, it records the current hookIndex. The hookIndex tracks the log index of the latest log entry that updated the hooks. As the hooks are stored in the repository at <repo>/wal/hooks/<log_index>, this is enough to determine the latest committed hooks in the repository at the time the transaction began. This is then exposed through the transaction which allows for calling code to invoke the correct hooks to do a snapshot read. This will eventually be piped to the HookManager which would use the value to invoke the correct hooks for the transaction, and should also be considered during other reads of the hooks such as backups.

The assumption is that hookIndex is always points to the latest hooks. This holds during normal operation as the hookIndex is always updated when new hooks are logged. To make this hold even after restarts, logic is added in initialize to determine the latest hooks by peeking in the write-ahead log and the repository.

The hooks are currently not pruned from the repository after being replaced, so the hooks are guaranteed to stay in place while the Transaction runs. Now that we can track which hooks each transaction is using, we can later in a follow up implement pruning of old hooks once there are no open transactions using them.

Part of #4792 (closed)

Merge request reports