PostReceiveHook invoked before write has been recorded by Praefect
In Gitaly Cluster, the primary replica executes the hook logic. With a replica set of three nodes, only the primary and one replica need to vote in order for a write for a vote to be considered successful. Once the vote is successful, the voters are unblocked and proceed applying the write, and the primary runs the PostReceiveHook. At this point, it's not guaranteed that all replicas have applied the write. The write also hasn't been recorded by Praefect in the database yet as that happens only after proxying the write to all replicas concludes.
This leads to a race where the primary triggers the PostReceiveHook before the write has been concluded. Rails may then trigger a pipeline for the write before all replicas have processed it and before the the write is considered by routing. This may then fail the pipeline as the requests made by the pipeline execution may target nodes that do not yet have the write.