Database Load balancing - Service Discovery thread caches IP of resolver

From gitlab-com/gl-infra/production#8097 (comment 1190374493), it looks like caches the IP address of the resolver:

1. LoadBalancing::Resolver#resolver returns the IP address of the consul-gl-consul-dns.consul.svc.cluster.local nameserver
2. In https://gitlab.com/gitlab-org/gitlab/-/blob/b254f573d248a845083da71f993e675ef0c154ad/lib/gitlab/database/load_balancing/service_discovery.rb#L200-202, the IP is passed as the nameservers arg of Net::DNS::Resolver.new. It is then memoized for the whole lifetime of the refresh thread.

Proposal

So I suggest we introduce a way to expire the Net::DNS::Resolver based on the DNS TTL

/cc @stomlinson @alexives @DylanGriffith @gsgl