Support DNS+TLS connections with Praefect
When there are multiple Praefects, DNS can be used for client-side load balancing. This can help balancing traffic across Praefect. However, this currently doesn't support TLS. The issue is that when the scheme `dns:[//authority_host[:authority_port]/]host[:port]` is used, the dns server resolves the IP and those IPs end up being used for the TLS handshake, which will fail because `[host]` is what's in the certificate. Instead, on the Rails/Workhorse/gitlab-shell side, we should configure SNI overrides when creating the gRPC connection so we can support dns and tls. ## Proposal Allow Rails to parse a `dns+tls://resolver/host:port` scheme. This will indicate tls will be involved, and allow us to pass in the `host` to override the SNI so that a TLS handshake succeeds between Praefect and its client. This diagram shows how Workhorse establishes TLS connections to Praefect nodes using the `dns+tls://` scheme. NOTE: without Workhorse in the picture, Rails will also use the SNI override directly make the TLS connection to Praefect ```mermaid graph TB subgraph Config["Rails Configuration"] Conf["gitaly_address:<br/>dns+tls://1.1.1.1/praefect.example.com:8075"] end subgraph API["GitLab API"] GitLabAPI["GET /api/v4/internal/authorize<br/>Returns: gitaly_address"] end subgraph WH["Workhorse (gRPC Client)"] WHClient[gRPC Client<br/>Round-robin Load Balancer] WHParser["Parse URI:<br/>• Resolver: 1.1.1.1<br/>• Host: praefect.example.com<br/>• Port: 8075<br/>• SNI: praefect.example.com"] end subgraph Shell["GitLab-Shell (gRPC Client)"] ShellClient[gRPC Client<br/>Round-robin Load Balancer] ShellParser["Parse URI:<br/>• Resolver: 1.1.1.1<br/>• Host: praefect.example.com<br/>• Port: 8075<br/>• SNI: praefect.example.com"] end subgraph DNS["DNS Resolver"] Resolver["1.1.1.1"] end subgraph Praefect["Praefect Cluster"] P1["Praefect Node 1<br/>10.0.1.10:8075<br/>Cert: praefect.example.com"] P2["Praefect Node 2<br/>10.0.1.11:8075<br/>Cert: praefect.example.com"] P3["Praefect Node 3<br/>10.0.1.12:8075<br/>Cert: praefect.example.com"] end %% Rails config to API Conf -->|"Stores config"| GitLabAPI %% Workhorse flow GitLabAPI -->|"1a. Returns gitaly_address"| WHParser WHParser -->|"2a. DNS Query:<br/>praefect.example.com"| Resolver Resolver -->|"3a. DNS Response:<br/>10.0.1.10, 10.0.1.11, 10.0.1.12"| WHClient %% Shell flow GitLabAPI -->|"1b. Returns gitaly_address"| ShellParser ShellParser -->|"2b. DNS Query:<br/>praefect.example.com"| Resolver Resolver -->|"3b. DNS Response:<br/>10.0.1.10, 10.0.1.11, 10.0.1.12"| ShellClient %% Workhorse to Praefect connections WHClient -->|"4a. TLS + gRPC<br/>SNI: praefect.example.com"| P1 WHClient -->|"4b. TLS + gRPC<br/>SNI: praefect.example.com"| P2 WHClient -->|"4c. TLS + gRPC<br/>SNI: praefect.example.com"| P3 %% Shell to Praefect connections ShellClient -->|"4d. TLS + gRPC<br/>SNI: praefect.example.com"| P1 ShellClient -->|"4e. TLS + gRPC<br/>SNI: praefect.example.com"| P2 ShellClient -->|"4f. TLS + gRPC<br/>SNI: praefect.example.com"| P3 %% Styling style Conf fill:#e8f4f8,stroke:#333,stroke-width:2px style Config fill:#e8f4f8,stroke:#333,stroke-width:2px style API fill:#fc6d26,stroke:#333,stroke-width:3px style GitLabAPI fill:#fd7e14,stroke:#333,stroke-width:2px,color:#fff style WH fill:#fca326,stroke:#333,stroke-width:3px style WHClient fill:#fdb863,stroke:#333,stroke-width:2px style WHParser fill:#fdb863,stroke:#333,stroke-width:2px style Shell fill:#2da44e,stroke:#333,stroke-width:3px style ShellClient fill:#3fb950,stroke:#333,stroke-width:2px style ShellParser fill:#3fb950,stroke:#333,stroke-width:2px style DNS fill:#6b4fbb,stroke:#333,stroke-width:3px style Resolver fill:#8b6fcb,stroke:#333,stroke-width:2px,color:#fff style Praefect fill:#1f75cb,stroke:#333,stroke-width:3px style P1 fill:#4a9eda,stroke:#333,stroke-width:2px,color:#fff style P2 fill:#4a9eda,stroke:#333,stroke-width:2px,color:#fff style P3 fill:#4a9eda,stroke:#333,stroke-width:2px,color:#fff ``` --- ## How DNS+TLS Works ### URI Format ``` dns+tls://[resolver]/[hostname]:[port] └─────┬────┘ └──────┬──────┘ │ │ DNS resolver Target service (optional) (required) ``` ### Example ``` dns+tls://1.1.1.1/praefect.example.com:8075 ``` **Parsed as:** - **Scheme**: `dns+tls` → Enables TLS + DNS resolution - **Authority**: `1.1.1.1` → DNS resolver to use - **Path**: `/praefect.example.com:8075` → Target hostname and port - **SNI Override**: `praefect.example.com` → Extracted from path for TLS ### Connection Flow 1. **Parse Configuration**: Extract resolver, hostname, port from URI 1. **Enable TLS**: `stub_creds()` detects `dns+tls` scheme 1. **Set SNI Override**: `channel_args()` extracts hostname from path 1. **DNS Resolution**: Query resolver for hostname → get multiple IPs 1. **Round-robin**: Distribute requests evenly across all subchannels 1. **TLS Handshake**: Each connection uses SNI for proper certificate validation
issue