Support DNS+TLS connections with Praefect
When there are multiple Praefects, DNS can be used for client-side load balancing. This can help balancing traffic across Praefect. However, this currently doesn't support TLS. The issue is that when the scheme `dns:[//authority_host[:authority_port]/]host[:port]` is used, the dns server resolves the IP and those IPs end up being used for the TLS handshake, which will fail because `[host]` is what's in the certificate.
Instead, on the Rails/Workhorse/gitlab-shell side, we should configure SNI overrides when creating the gRPC connection so we can support dns and tls.
## Proposal
Allow Rails to parse a `dns+tls://resolver/host:port` scheme. This will indicate tls will be involved, and allow us to pass in the `host` to override the SNI so that a TLS handshake succeeds between Praefect and its client.
This diagram shows how Workhorse establishes TLS connections to Praefect nodes using the `dns+tls://` scheme.
NOTE: without Workhorse in the picture, Rails will also use the SNI override directly make the TLS connection to Praefect
```mermaid
graph TB
subgraph Config["Rails Configuration"]
Conf["gitaly_address:<br/>dns+tls://1.1.1.1/praefect.example.com:8075"]
end
subgraph API["GitLab API"]
GitLabAPI["GET /api/v4/internal/authorize<br/>Returns: gitaly_address"]
end
subgraph WH["Workhorse (gRPC Client)"]
WHClient[gRPC Client<br/>Round-robin Load Balancer]
WHParser["Parse URI:<br/>• Resolver: 1.1.1.1<br/>• Host: praefect.example.com<br/>• Port: 8075<br/>• SNI: praefect.example.com"]
end
subgraph Shell["GitLab-Shell (gRPC Client)"]
ShellClient[gRPC Client<br/>Round-robin Load Balancer]
ShellParser["Parse URI:<br/>• Resolver: 1.1.1.1<br/>• Host: praefect.example.com<br/>• Port: 8075<br/>• SNI: praefect.example.com"]
end
subgraph DNS["DNS Resolver"]
Resolver["1.1.1.1"]
end
subgraph Praefect["Praefect Cluster"]
P1["Praefect Node 1<br/>10.0.1.10:8075<br/>Cert: praefect.example.com"]
P2["Praefect Node 2<br/>10.0.1.11:8075<br/>Cert: praefect.example.com"]
P3["Praefect Node 3<br/>10.0.1.12:8075<br/>Cert: praefect.example.com"]
end
%% Rails config to API
Conf -->|"Stores config"| GitLabAPI
%% Workhorse flow
GitLabAPI -->|"1a. Returns gitaly_address"| WHParser
WHParser -->|"2a. DNS Query:<br/>praefect.example.com"| Resolver
Resolver -->|"3a. DNS Response:<br/>10.0.1.10, 10.0.1.11, 10.0.1.12"| WHClient
%% Shell flow
GitLabAPI -->|"1b. Returns gitaly_address"| ShellParser
ShellParser -->|"2b. DNS Query:<br/>praefect.example.com"| Resolver
Resolver -->|"3b. DNS Response:<br/>10.0.1.10, 10.0.1.11, 10.0.1.12"| ShellClient
%% Workhorse to Praefect connections
WHClient -->|"4a. TLS + gRPC<br/>SNI: praefect.example.com"| P1
WHClient -->|"4b. TLS + gRPC<br/>SNI: praefect.example.com"| P2
WHClient -->|"4c. TLS + gRPC<br/>SNI: praefect.example.com"| P3
%% Shell to Praefect connections
ShellClient -->|"4d. TLS + gRPC<br/>SNI: praefect.example.com"| P1
ShellClient -->|"4e. TLS + gRPC<br/>SNI: praefect.example.com"| P2
ShellClient -->|"4f. TLS + gRPC<br/>SNI: praefect.example.com"| P3
%% Styling
style Conf fill:#e8f4f8,stroke:#333,stroke-width:2px
style Config fill:#e8f4f8,stroke:#333,stroke-width:2px
style API fill:#fc6d26,stroke:#333,stroke-width:3px
style GitLabAPI fill:#fd7e14,stroke:#333,stroke-width:2px,color:#fff
style WH fill:#fca326,stroke:#333,stroke-width:3px
style WHClient fill:#fdb863,stroke:#333,stroke-width:2px
style WHParser fill:#fdb863,stroke:#333,stroke-width:2px
style Shell fill:#2da44e,stroke:#333,stroke-width:3px
style ShellClient fill:#3fb950,stroke:#333,stroke-width:2px
style ShellParser fill:#3fb950,stroke:#333,stroke-width:2px
style DNS fill:#6b4fbb,stroke:#333,stroke-width:3px
style Resolver fill:#8b6fcb,stroke:#333,stroke-width:2px,color:#fff
style Praefect fill:#1f75cb,stroke:#333,stroke-width:3px
style P1 fill:#4a9eda,stroke:#333,stroke-width:2px,color:#fff
style P2 fill:#4a9eda,stroke:#333,stroke-width:2px,color:#fff
style P3 fill:#4a9eda,stroke:#333,stroke-width:2px,color:#fff
```
---
## How DNS+TLS Works
### URI Format
```
dns+tls://[resolver]/[hostname]:[port]
└─────┬────┘ └──────┬──────┘
│ │
DNS resolver Target service
(optional) (required)
```
### Example
```
dns+tls://1.1.1.1/praefect.example.com:8075
```
**Parsed as:**
- **Scheme**: `dns+tls` → Enables TLS + DNS resolution
- **Authority**: `1.1.1.1` → DNS resolver to use
- **Path**: `/praefect.example.com:8075` → Target hostname and port
- **SNI Override**: `praefect.example.com` → Extracted from path for TLS
### Connection Flow
1. **Parse Configuration**: Extract resolver, hostname, port from URI
1. **Enable TLS**: `stub_creds()` detects `dns+tls` scheme
1. **Set SNI Override**: `channel_args()` extracts hostname from path
1. **DNS Resolution**: Query resolver for hostname → get multiple IPs
1. **Round-robin**: Distribute requests evenly across all subchannels
1. **TLS Handshake**: Each connection uses SNI for proper certificate validation
issue