Synchronize bundle generation between Praefect nodes

Context

In the context of bundle URI, when generating bundles automatically, Gitaly keeps track of the last time it generated a bundle for this particular repository. This can be seen in action here. The state of each bundles generated that Gitaly keeps track of are in memory.

Issue

When Gitaly runs in non-Praefect mode, that is a non-issue.

However, when Gitaly runs in Praefect mode, where repositories are replicated across nodes, that is an issue, because the in-memory state is not shared between node.

Let's say, we have 3 nodes: A, B and C. If node A generates a bundle, then node A knows when that bundle was created (because it keep tracks of it in memory), but node B and C has no way of knowing that a bundle was already generated. This can lead to multiple nodes generating bundles unnecessarily.

The solution is to use the IfNotExist field in the Sink writer options here.

However, merely using this field will break usage for manual bundle generation. This might prevent a manual bundle generation command from succeeding if a bundle already exist.

What to do

This issue is about using conditional writes for automatic bundle generation (ie: the IfNotExist field set to true), while at the same time allowing overwriting bundles for manual bundle generation.