Investigate kas long connection timeout
From #255 (comment 1081803797):
The fix has just been rolled out to GitLab.com. I've tested it here https://gitlab.com/ash2k/agents-config/-/jobs/2951488844 and the behavior is better but the problem still exists. It took 10 minutes for the error to appear, which is a big improvement over 15 seconds. It's likely a timeout in yet another place. I'll investigate further.
I0830 05:54:43.738326 28 round_trippers.go:553] GET https://kas.gitlab.com/k8s-proxy/api/v1/namespaces/kube-system/pods/coredns-565d847f94-b8qlx/log?container=coredns&follow=true 200 OK in 271 milliseconds
I0830 05:54:43.738707 28 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 271 ms Duration 271 ms
I0830 05:54:43.738903 28 round_trippers.go:577] Response Headers:
I0830 05:54:43.739078 28 round_trippers.go:580] Via: 2.0 gitlab-agent/v0.0.0/00000000
I0830 05:54:43.739255 28 round_trippers.go:580] Via: gRPC/1.0 gitlab-kas/v15.4.0/b37e04c, 1.1 google
I0830 05:54:43.739386 28 round_trippers.go:580] Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
I0830 05:54:43.739650 28 round_trippers.go:580] Audit-Id: e0d6769c-ac93-4287-b859-f38f73aa1234
I0830 05:54:43.739816 28 round_trippers.go:580] Cache-Control: no-cache, private
I0830 05:54:43.739996 28 round_trippers.go:580] Content-Type: text/plain
I0830 05:54:43.740171 28 round_trippers.go:580] Date: Tue, 30 Aug 2022 05:54:43 GMT
.:53
[INFO] plugin/reload: Running configuration SHA512 = 53330204b5b8aeef5f056d933359bc7e95d66a7fa922f7eef2e4e80e0962233039fd776606db297a43f0847fef9243d143e67037cf901385bcb310c27712d2c1
CoreDNS-1.9.3
linux/arm64, go1.18.2, 45b0a11
error: stream error: stream ID 3; INTERNAL_ERROR; received from peer
Tue Aug 30 06:04:35 UTC 2022
Two following runs also show the error after 9-10 minutes.
Edited by Mikhail Mazurskiy