Skip to content
GitLab
Next
    • Why GitLab
    • Pricing
    • Contact Sales
    • Explore
  • Why GitLab
  • Pricing
  • Contact Sales
  • Explore
  • Sign in
  • Get free trial
  • GitLab.orgGitLab.org
  • GitLabGitLab
  • Issues
  • #2042

Service discovery for database load balancing

Instead of manually configuring all secondaries to balance load amongst, we need a service discovery system. This system works basically as follows:

  1. Look up a DNS record for the primary. This record will contain the host to connect to for writes.
  2. Look up a DNS record for all secondaries. This record will return multiple values, each specifying a secondary to connect to.
  3. Configure ActiveRecord using the above settings

To handle refreshing we'll have a thread re-query this data every N minutes (not sure yet what a good default would be), with a small random value added to spread load a bit. If this thread notices the list of hosts is different it will:

  1. Reconfigure ActiveRecord using the new hosts
  2. Disconnect all existing connections

A hard disconnect might result in errors, or result in a request still trying to use old hosts (e.g. by immediately trying to reconnect). As such we probably need some kind of barrier/safe point before reconnecting.

This also means having to change the configuration format. This should be clearly documented as users will need to change database.yml accordingly for otherwise load balancing will be disabled.

Assignee
Assign to
Time tracking