Open source the database load balancer
Similar to https://gitlab.com/gitlab-org/gitlab-ce/issues/49465, @DouweM and I talked a bit about the database load balancer. In particular, I feel that the database load balancer should be open sourced, because it can be a very valuable piece of technology for everybody that is using Ruby on Rails. The only close alternative is https://github.com/thiagopradi/octopus, but it is largely aimed at sharding. It also has the following differences:
- In Octopus, using a secondary is opt-in, while in GitLab it's opt-out. opt-out makes things much easier
- Because of the opt-in nature, Octopus inherently requires more work. GitLab's solution on the other hand is a "plug and play" solution
- Octopus' sticking support is very limited, and last I checked it uses a cookie with a fixed 30 second timeout. This doesn't work for GitLab because we have to support sticking for clients that don't use cookies, such as CI runners.
Open sourcing the load balancer would give us a few benefits:
- Everybody can benefit from it, which I think fits nicely with GitLab's values and nature.
- The open source community can contribute back to it, e.g. by fixing bugs
- We decouple the code from GitLab, which in theory could result in nicer code
For this to work, we do have to make it very clear that it will never support anything but PostgreSQL, as we simply can't maintain anything that supports multiple databases (since we only use PostgreSQL).
This does bring the question: how would this affect our offering of this feature? With the Gem being open source, anybody could include it in CE. I'm thinking we should do the following in that case:
- We include the Gem and configuration in CE, saving us some merge conflicts
- We only provide support for GitLab with the load balancer if you use EE ultimate
From our perspective I expect it will take a bit of effort to extract the code, but overall it's not very GitLab specific at all, though it does require/depend on doing https://gitlab.com/gitlab-org/gitlab-ce/issues/49465 first. Once extracted, I doubt there will be much work left to do, as the code overall is fairly feature complete.