Handle repository creations, deletions and renames atomically

Renaming, creating and deleting repositories is racy in Praefect. They can also partially fail in awkward ways due to Praefect first applying the operations on the disks of the Gitalys and later updating its metadata store. In order to make these operations atomic, there's been ongoing work in the background to make it possible to perform these in the database in a manner that races are not possible and partial failures do not end up conflicting with future operations.

Renames can be fixed by doing the rename atomically in the database without moving any repositories on the disks.

Deletes can be modeled as simply deleting the repository's database record.

Creates are atomic by creating the repository's database record as the last step in the process.

The last piece of the puzzle is to ensure repositories always land in different directories on the disk. This ensures that a partial failure doesn't block a future operation from succeeding. This commit implements that piece by deriving a unique path for each repository to store their replicas. Existing repositories stay where they are but newly created repositories will land in unique directories.

Create might succeed on a disk but fail to be persisted in the database. Prior to this change, attempting to recreate a repository might fail due to the stale state on the disk. With this in place, the next attempt at creating the repository would still succeed as the new attempt would land the repository in a different directory on the disk.

Deletes have the same problem prior to this commit. The repository's metadata may be successfully deleted but if we fail to delete the repository from the disk, future creations and renames may fail if they conflict with the stale state. As creates now always land in a different directory, the stale state no longer causes problems.

Renames will work purely in the database, so any stale state on the disk will not affect them.

Closes #3485 (closed)
Closes #4003 (closed)

internal/praefect/coordinator.go

@@ -89,7 +89,6 @@ var transactionRPCs = map[string]transactionsCondition{
 	"/gitaly.ObjectPoolService/DisconnectGitAlternates":    transactionsDisabled,
 	"/gitaly.ObjectPoolService/LinkRepositoryToObjectPool": transactionsDisabled,
 	"/gitaly.ObjectPoolService/ReduplicateRepository":      transactionsDisabled,
-	"/gitaly.RepositoryService/RenameRepository":           transactionsDisabled,
 
 	// The following list of RPCs are considered idempotent RPCs: while they write into the
 	// target repository, this shouldn't ever have any user-visible impact given that they're
@@ -172,12 +171,6 @@ func getReplicationDetails(methodName string, m proto.Message) (datastore.Change
 		"/gitaly.RepositoryService/CreateRepositoryFromURL",
 		"/gitaly.RepositoryService/ReplicateRepository":
 		return datastore.CreateRepo, nil, nil
-	case "/gitaly.RepositoryService/RenameRepository":
-		req, ok := m.(*gitalypb.RenameRepositoryRequest)
-		if !ok {
-			return "", nil, fmt.Errorf("protocol changed: for method %q expected message type '%T', got '%T'", methodName, req, m)
-		}
-		return datastore.RenameRepo, datastore.Params{"RelativePath": req.RelativePath}, nil
 	case "/gitaly.RepositoryService/GarbageCollect":
 		req, ok := m.(*gitalypb.GarbageCollectRequest)
 		if !ok {
@@ -1037,17 +1030,6 @@ func (c *Coordinator) newRequestFinalizer(
 			if err := c.rs.IncrementGeneration(ctx, repositoryID, primary, updatedSecondaries); err != nil {
 				return fmt.Errorf("increment generation: %w", err)
 			}
-		case datastore.RenameRepo:
-			// Renaming a repository is not idempotent on Gitaly's side. This combined with a failure here results in a problematic state,
-			// where the client receives an error but can't retry the call as the repository has already been moved on the primary.
-			// Ideally the rename RPC should copy the repository instead of moving so the client can retry if this failed.
-			if err := c.rs.RenameRepository(ctx, virtualStorage, targetRepo.GetRelativePath(), primary, params["RelativePath"].(string)); err != nil {
-				if !errors.Is(err, datastore.RepositoryNotExistsError{}) {
-					return fmt.Errorf("rename repository: %w", err)
-				}
-
-				ctxlogrus.Extract(ctx).WithError(err).Info("renamed repository does not have a store entry")
-			}
 		case datastore.CreateRepo:
 			repositorySpecificPrimariesEnabled := c.conf.Failover.ElectionStrategy == config.ElectionStrategyPerRepository
 			variableReplicationFactorEnabled := repositorySpecificPrimariesEnabled &&