Skip to content

Running in HA mode (Praefect) generates a bundle per node

While testing cloning with bundles locally, I noticed that when running in HA mode (Praefect + 3 Gitaly nodes), one bundle is generate per node, because of the replication logic that calls the same RPC (PostUploadPackWithSidechannel) on each node. This is very inefective.

There are some solutions, but they each have drawbacks.

  1. We could only set a go_cloud_url (thus enabling the generation of bundles) config on the configuration of the primary node only. That would in effect make sure only the primary node generate bundles.
    1. However, I am still not well versed into how failover works, but if we have automatic failover for Gitaly nodes, this solution is bad because the new primary node won't have the go_cloud_url config set.
    2. The bundle path has the storage name in it (see here). So currently, a Gitaly node is unable to know if a bundle created by another Gitaly node exists for a given repository. We would need to make it so that a bundle path is common across all Gitaly nodes, regardless of the storage name.
  2. Move the bundle-generation into Praefect
    1. Since Praefect coordonnates the different Gitaly nodes, it would have all the knowledge it needs to make sure to create a bundle one once per git clone or git fefch .
    2. If I understood correctly, I think the goal of the Raft initiative is to move away from Praefect. If that is the case, this solution is probably a no-go then.

Those are the solutions I have so far. Any feedback or suggestions welcome!

Edited by Olivier Campeau
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information