Skip to content

Don't assign non-existent repositories to partitions

Sami Hiltunen requested to merge smh-non-existing-assignment into master

Transactions require repositories to belong to a partition in order to start a transaction against the correct partition. All transactions for a given repository must go through the same partition to ensure the transactional guarantees. When we're migrating to transactions, the existing repositories haven't yet been assigned to partitions. Gitaly thus assigns a repository to a partition when it's first accessed. Currently it does so also for non-existent repositories to ensure all RPCs targeting such a repository target the same partition. This has a few problems:

  1. Accessing a non-existent repository creates a partition assignment record. That record will be stale if the repository is never created.
  2. Rails is sending RepositoryExists requests against a to-be-created fork prior to creating it. This in turn assigns it into a partition. As the RepositoryExists request doesn't contain any information about the origin or the object pool we'd like to partition the fork with, we'll end up placing the fork into the wrong partition.

This commits solves the above problems by erroring out when accessing a non-existent repository. This ensures we don't create partition assignments for non-existent repositories nor assign the forks into incorrect partitions.

Repositories that exist on the disk are still assigned into a partition as before on first access.

If a repository does not exist yet but the RPC is a repository creating one, we'll create the partition assignment. This enables us to start the transaction that will create the repository ultimately.

Handling for RepositoryExists is special cased. We can't launch a transaction against a repository without a partition assignment. If the repository doesn't exist, we won't create one. This prevents us from calling the handler with a transaction. The interface of the RPC is to return a successful response with a boolean indicating whether or not the repository existed. We handle the new error raised for non-existent repositories by returning the same response as the handler would if the repository didn't exist.

Some tests had to be adjusted to account for the 'repository not found' error returned from the middleware. The wording of the error and the metadata may differ with the exact error returned by some of the RPC handlers.

Closes #5957 (closed)
Closes #5887 (closed)

Edited by Sami Hiltunen

Merge request reports