Pushes using transactions fail with reads distribution
When pushing into a repository, git-receive-pack creates a quarantine environment for the received objects so they don't end up in the repository if the push is rejected at some point. This quarantine environment is generated with a random name and is neither predictable nor can its name be changed.
This quarantine environment needs to be made available to all git commands which shall examine this push, which is done by adding it to the GIT_OBJECT_DIRECTORIES environment variable. The most important user of this is the /allowed
GitLab Rails API which is invoked by the pre-receive hook in order to see whether a given push is allowed or not. To make it aware of the quarantine environment, we explicitly POST the AllowedParams
, which besides other things includes the value of the push quarantine object directory. Any subsequent calls to Gitaly to inspect the push thus make sure to pass this value to Gitaly correctly such that it'll know to use the quarantined object directory.
With transactions enabled, we now have the situation that each of the Gitaly members which are part of the push will have generated a different name for the quarantine directory. Which is fine as long as we only route requests to the primary node. And because of #2977 (closed), we always did even if reads distribution was enabled because by the time any accessors get routed to us to inspect the push, the repository generation was already incremented on the primary node because of the Cleanup
call. Secondaries are thus treated as out-of-date and won't get any accessors routed to them anymore.
This breaks as soon as #2977 (closed) is fixed though because now the repo will be considered up-to-date on all nodes during the whole push operation. Reads distribution will now kick in and route the accessor to a random node with the quarantine environment of the primary. And as secondaries have a different quarantine environment, they request will fail and cause the push to be canceled with a "Not allowed" error message.