Draft: chore(database): add consul API pkg for service discovery

Related to Add configuration to use service discovery to i... (#890 - closed)

DB Replication setup with GDK

In order to test locally, you can setup database replication tweaking and following the GDK guide to setup load balancing with service discovery. Here are the steps.

1. Follow the local database setup with the GDK guide.
2. From the GDK directory, add the following line to support/templates/pgbouncer/pgbouncer-replica.ini.erb:

diff --git a/support/templates/pgbouncer/pgbouncer-replica.ini.erb b/support/templates/pgbouncer/pgbouncer-replica.ini.erb
index 62e53df3..c663e9d5 100644
--- a/support/templates/pgbouncer/pgbouncer-replica.ini.erb
+++ b/support/templates/pgbouncer/pgbouncer-replica.ini.erb
@@ -1,6 +1,7 @@
 [databases]
 gitlabhq_development = host=<%= host %> dbname=gitlabhq_development user=<%= config.__whoami %>
 gitlabhq_development_ci = host=<%= host %> dbname=gitlabhq_development_ci user=<%= config.__whoami %>
+registry_dev = host=<%= host %> dbname=registry_dev user=<%= config.__whoami %>

 [pgbouncer]
 listen_port = <%= port %>

3. Optional. Add a check to the consul config file support/templates/consul/config.json.erb

diff --git a/support/templates/consul/config.json.erb b/support/templates/consul/config.json.erb
index 83399cb4..94da8233 100644
--- a/support/templates/consul/config.json.erb
+++ b/support/templates/consul/config.json.erb
@@ -16,6 +16,10 @@
         {
           "args": ["psql", "-U", "<%= config.__whoami %>", "-h", "localhost", "-p", "<%= port %>", "-d", "gitlabhq_development", "-c", "select 1"],
           "interval": "2s"
+        },
+         {
+          "args": ["psql", "-U", "<%= config.__whoami %>", "-h", "localhost", "-p", "<%= port %>", "-d", "registry_dev", "-c", "select 1"],
+          "interval": "2s"
         }
       ]
     }

4. Follow the load balancing with service discovery guide in the GDK. Make sure to run the first part!

5. On the registry side, export the following variable. Note this is the default service name locally for the replicas, there is no service discovery for the primary so this is just meant to test the connection to consul.

 export REGISTRY_CONSUL_DB_PRIMARY_SERVICE=pgbouncer

6. Build the registry and run database command, for example check the migration status and look for a similar output:

make bin/registry && ./bin/registry database migrate status -u ./myenv/config.yml
+ bin/registry
WARN[0000] ignoring unrecognized environment variable    go_version=go1.20 name=REGISTRY_CONSUL_DB_PRIMARY_SERVICE version=v3.69.0-gitlab-19-g51868d65
DNS query ID: "42d89faa-265e-49dd-f68c-a6893ae46cfb" for service: "pgbouncer"
got service record: &{Kind: ID:pgbouncer-6432 Service:pgbouncer Tags:[replica] Meta:map[] Port:6432 Address:127.0.0.1 SocketPath: TaggedAddresses:map[lan_ipv4:{Address:127.0.0.1 Port:6432} wan_ipv4:{Address:127.0.0.1 Port:6432}] Weights:{Passing:1 Warning:1} EnableTagOverride:false CreateIndex:18 ModifyIndex:18 ContentHash: Proxy:0x140005dd380 Connect:0x140004c8c80 PeerName: Namespace: Partition: Datacenter:}
got service record: &{Kind: ID:pgbouncer-6435 Service:pgbouncer Tags:[replica] Meta:map[] Port:6435 Address:127.0.0.1 SocketPath: TaggedAddresses:map[lan_ipv4:{Address:127.0.0.1 Port:6435} wan_ipv4:{Address:127.0.0.1 Port:6435}] Weights:{Passing:1 Warning:1} EnableTagOverride:false CreateIndex:21 ModifyIndex:21 ContentHash: Proxy:0x140005dd520 Connect:0x140004c8d50 PeerName: Namespace: Partition: Datacenter:}
got service record: &{Kind: ID:pgbouncer-6434 Service:pgbouncer Tags:[replica] Meta:map[] Port:6434 Address:127.0.0.1 SocketPath: TaggedAddresses:map[lan_ipv4:{Address:127.0.0.1 Port:6434} wan_ipv4:{Address:127.0.0.1 Port:6434}] Weights:{Passing:1 Warning:1} EnableTagOverride:false CreateIndex:20 ModifyIndex:20 ContentHash: Proxy:0x140005dd5f0 Connect:0x140004c8df0 PeerName: Namespace: Partition: Datacenter:}
got service record: &{Kind: ID:pgbouncer-6433 Service:pgbouncer Tags:[replica] Meta:map[] Port:6433 Address:127.0.0.1 SocketPath: TaggedAddresses:map[lan_ipv4:{Address:127.0.0.1 Port:6433} wan_ipv4:{Address:127.0.0.1 Port:6433}] Weights:{Passing:1 Warning:1} EnableTagOverride:false CreateIndex:19 ModifyIndex:19 ContentHash: Proxy:0x140005dd6c0 Connect:0x140004c8eb0 PeerName: Namespace: Partition: Datacenter:}
true

What's next?

The intent here is to deploy the change to test the consul connection in our different environments, most importantly in gstg and gprd. We should then in the logs for the output, hopefully confirming that we got a record for the correct service.

• Question: Since the migrations run in a separate container in the registry pod, it's possible that the log will not show up in Kibana. If that's the case, we can either get someone from infra to run the status command manually. Alternatively, we can execute the same function on service startup so we can inspect the logs in Kibana.