Skip to content

WIP: POC 3phase writeref

John Cai requested to merge jc-3phase-writeref into master

This MR has several components in order to do a 3phase commit for WriteRef

  • Add a custom stream handler in praefect that handles WriteRef
  • Add a central lock whereby repositories can be locked while transactions can be created and committed

What this leaves out:

  • serialization of transactions

fixes: #2466 (closed)

Here is the overall call structure.

sequenceDiagram
  Rails->>Praefect: write_ref(repo1, ref, new_rev, old_rev)
  Praefect->>InternalGitaly_0: vote - is repo1's ref at old_rev?
  activate InternalGitaly_0
  Praefect->>InternalGitaly_1: vote - is repo1's ref at old_rev?
  activate InternalGitaly_1
  Praefect->>InternalGitaly_2: vote - is repo1's ref at old_rev?
  activate InternalGitaly_2
  InternalGitaly_0->>Praefect: yes
  InternalGitaly_1->>Praefect: yes
  InternalGitaly_2->>Praefect: yes
  Praefect->>InternalGitaly_0: precommit
  Praefect->>InternalGitaly_1: precommit
  Praefect->>InternalGitaly_2: precommit
  InternalGitaly_0->>Praefect: success
  InternalGitaly_1->>Praefect: success
  InternalGitaly_2->>Praefect: success
  Praefect->>InternalGitaly_0: commit
  InternalGitaly_0->>Praefect: success
  deactivate InternalGitaly_0
  Praefect->>InternalGitaly_1: commit
  InternalGitaly_1->>Praefect: success
  deactivate InternalGitaly_1
  Praefect->>InternalGitaly_2: commit
  InternalGitaly_2->>Praefect: success
  deactivate InternalGitaly_2
  Praefect->>Rails: success

Within an internal gitaly node, this is how a transaction works.

NOTE: the "prepare" state is not a separate call, but is essentially accomplished through putting a lock on the repo

sequenceDiagram
  Praefect->>Gitaly(TransactionServerInterceptor): WriteRefTx(repo1, step: precommit)
  Gitaly(TransactionServerInterceptor)->>TransactionManager: Lock repo1
  TransactionManager->>Gitaly(TransactionServerInterceptor): OK
  Gitaly(TransactionServerInterceptor)->>TransactionManager: New transaction abcd1234
  TransactionManager->>Gitaly(TransactionServerInterceptor): OK
  Gitaly(TransactionServerInterceptor)->>WriteRefTx: is repo1's ref at oldrev?
  WriteRefTx->>TransactionManager: Begin Transaction abcd1234
  TransactionManager->>WriteRefTx: OK
  WriteRefTx->>Git: git -C repo1 rev-parse ref
  Git->>WriteRefTx: Exit code 0
  WriteRefTx->>Gitaly(TransactionServerInterceptor): response(OK)
  Gitaly(TransactionServerInterceptor)->>Praefect: response(OK)
  Praefect->>Gitaly(TransactionServerInterceptor): WriteRefTx(repo1, step: commit)
  Gitaly(TransactionServerInterceptor)->>WriteRefTx: update ref from oldrev to rev
  WriteRefTx->>Git: git -C repo1 update-ref ref rev oldrev
  Git->>WriteRefTx: Exit code 0
  WriteRefTx->>Gitaly(TransactionServerInterceptor): response(OK)
  Gitaly(TransactionServerInterceptor)->>TransactionManager: End transaction abcd1234
  TransactionManager->>Gitaly(TransactionServerInterceptor): OK
  Gitaly(TransactionServerInterceptor)->>TransactionManager: Unlock repo1
  TransactionManager->>Gitaly(TransactionServerInterceptor): OK
  Gitaly(TransactionServerInterceptor)->>Praefect: response(OK)
Edited by John Cai

Merge request reports