Distribute kas to different regions to improve CI tunnel latency
agentk connects to a kas instance and then a request-to-be-proxied lands on the same or another kas instance. It'd be preferable to route both requests via kas instances that are closest to the source of the request. That way request from agentk in e.g. Australia and request from a CI job on a runner in Australia would not go to US (for GitLab.com), but stay in Australia. That means less latency + likely higher throughput (because less latency means TCP can have more data transmitted per second with the same window size. This is because of https://en.wikipedia.org/wiki/Bandwidth-delay_product).
To do the above, we need:
- kas needs to be deployed in all major (latency-wise) locations on the planet.
- HAproxy needs to be deployed in those same locations with routing to local kas instances only.
- Load balancing needs to be configured to do latency-based routing. CloudFlare probably does this already, but then we have HAProxy.
No code changes should be required as kas already prefers kas instances that have/had connections from an agent id that is being looked for. So, if agent is in Australia it'd connect to kas in Australia and that kas (and it's replicas in the same region) would route via that local group.