&8911 introduced write-ahead logging and transaction management logic in Gitaly. While the core implementation is actively being worked on, it's functional and has a mostly stable interface. This means we can begin integrating it in Gitaly for testing. Below are the steps to take to integrate the WAL logic into an RPC handler. # Integration The entry point into the transaction logic is the [`PartitionManager`](https://gitlab.com/gitlab-org/gitaly/-/blob/master/internal/gitaly/partition_manager.go#L23). If it is not yet available in the server instance, it needs to be wired there. The WAL will initially be configurable to roll it out safely. If the WAL is enabled, `PartitionManager` will be configured and injected as a dependency. All users of `PartitionManager` must check first whether `PartitionManager != nil`. If it is set, the WAL is enabled and the WAL logic should be ran. If it isn't set, the current non-WAL logic should be run. All reads and writes into repositories must happen within a transaction to guarantee the transactional semantics. Transactions are started with `PartitionManager.Begin()`. Every transaction must always be either rolled back or committed with `Transaction.Rollback()` or `Transaction.Commit()`. This releases the resources associated with the transition. If the transaction must read or execute hooks, it must be done by using the hooks residing at `tx.Snapshot().CustomHookPath`. This ensures the hooks the operations use the hooks in the read snapshot. The various write operations can be done by staging the writes in the transaction, and then at the end calling `Transaction.Commit()`. - `tx.UpdateReferences()` can be used to set the reference updates in the transaction - `tx.QuarantineDirectory()` is used for writing objects. The returned quarantine directory must be configured on the git invocations that write objects. - `tx.SetDefaultBranch()` can be used for setting the default branch. - `tx.SetCustomHooks()` is used for writing custom hooks in to the repository. - `tx.DeleteRepository()` can be called to delete the repository on commit. An RPC is fully integrated once - Reads: - All reads from a repository happen within a transaction - Any hook reads are done using the path from the snapshot - Writes - All objects are written into the transaction's quarantine directory. - Custom hooks, reference updates, default branch updates and deletions all are done through the transaction. - Hooks are executed from the path in the snapshot. - No writes are done to the repository directly, only through the transaction. ### Tests The `PartitionManager` is wired to the Gitaly test server when the `GITALY_TEST_WAL` environment variable is set. This environment variable is set in the`make test-wal` and `make test-praefect-with-wal` targets. The CI jobs `test:wal` and `test:test-with-praefect-wal` run the tests with the WAL enabled. While the external behavior should remain generally the same, tests may need some attention while integrating. - We must ensure that voting behavior with Praefect remains the same between the WAL enabled or not. Currently Gitaly tests do not generally assert the vote content which allows for tests to pass even if the votes have changed. We should update the RPCs tests to assert the complete content of the votes. See the [update to WriteRef tests](https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5812/diffs?commit_id=8d334d64013cd117429182e1b2afdf37f81b2a5d) for example. - We should test as much as possible through the API to ensure the tests remain agnostic to the storage details. This ensures we have flexibility in modifying the storage implementation behind an API. - While integrating, the tests may fail due to asserting disk state rather than behavior or state through the API. For example, the custom hooks are stored in a different location with the WAL enabled than without WAL. We should address these failures by updating the tests to assert the state and behavior through the API. This runs them through the transaction management and makes the tests agnostic to the storage details. ## Examples ### Reads ``` func (s *Server) handler(ctx context.Context, req *Request) error { tx, err := s.partitionManager.Begin(ctx, req.Repository) if err != nil { return fmt.Errorf("begin: %w", err) } // Defer the rollback so the transaction will be cleaned up even if it isn't committed. defer txutil.LogRollback(ctx, tx) // Do any thing that needs to be done. The open transaction guarantees the data in the read snapshot stays present. // If reading hooks, they must be read from the path in the snapshot. This ensures correct version of the hooks is read. hookPath := tx.Snapshot().CustomHookPath // Reads don't need to be committed since they're not doing any changes. The deferred rollback will clean up after // the transaction. return nil } ``` ### Writes ``` func (s *Server) handler(ctx context.Context, req *Request) error { tx, err := s.partitionManager.Begin(ctx, req.Repository) if err != nil { return fmt.Errorf("begin: %w", err) } // Defer the rollback so the transaction will be cleaned up even if it isn't committed. defer txutil.LogRollback(ctx, tx) // Do any thing that needs to be done. The open transaction guarantees the data in the read snapshot stays present. // If hooks should be executed, they must be executed from the path in the snapshot. This ensures correct version of the // hooks is executed. This is mostly likely already handled by passing the transaction to the HookManager. hookPath := tx.Snapshot().CustomHookPath // If the transaction needs to write objects, the objects must be written into the quarantine directory of the transaction. quarantineDir, _ := tx.QuarantineDirectory() unquarantinedRepo := s.localrepo(req.Repository) quarantinedRepo, _ := unquarantinedRepo.Quarantine(quarantineDir) quarantinedRepo.WriteBlob(...) // Update references by setting them as below. Reference updates are verified on commit. tx.UpdateReferences(gitaly.ReferenceUpdates{ "refs/heads/main": {OldOID: <oid>, NewOID: <oid>}, // If Force is set, old value of the reference is ignored. "refs/heads/branch" {Force: true, NewOID: <oid>}, }) // Set the default branch in the transaction. tx.SetDefaultBranch("refs/heads/main") // Set custom hooks in the transaction. The TAR should follow the same structure as what is currently // sent via `SetCustomHooks` RPC. tx.SetCustomHooks(<TAR>) // Delete the repository. tx.DeleteRepository() // None of the above modifications are done before the transaction is committed. Once Commit returns successfully // the changes have been successfully committed. if err := tx.Commit(ctx); err != nil { return fmt.Errorf("commit: %w", err) } return nil } ```