Investigate kas long connection timeout
From https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/issues/255#note_1081803797: The fix has just been rolled out to GitLab.com. I've tested it here https://gitlab.com/ash2k/agents-config/-/jobs/2951488844 and the behavior is better but the problem still exists. It took 10 minutes for the error to appear, which is a big improvement over 15 seconds. It's likely a timeout in yet another place. I'll investigate further. ``` I0830 05:54:43.738326 28 round_trippers.go:553] GET https://kas.gitlab.com/k8s-proxy/api/v1/namespaces/kube-system/pods/coredns-565d847f94-b8qlx/log?container=coredns&follow=true 200 OK in 271 milliseconds I0830 05:54:43.738707 28 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 271 ms Duration 271 ms I0830 05:54:43.738903 28 round_trippers.go:577] Response Headers: I0830 05:54:43.739078 28 round_trippers.go:580] Via: 2.0 gitlab-agent/v0.0.0/00000000 I0830 05:54:43.739255 28 round_trippers.go:580] Via: gRPC/1.0 gitlab-kas/v15.4.0/b37e04c, 1.1 google I0830 05:54:43.739386 28 round_trippers.go:580] Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 I0830 05:54:43.739650 28 round_trippers.go:580] Audit-Id: e0d6769c-ac93-4287-b859-f38f73aa1234 I0830 05:54:43.739816 28 round_trippers.go:580] Cache-Control: no-cache, private I0830 05:54:43.739996 28 round_trippers.go:580] Content-Type: text/plain I0830 05:54:43.740171 28 round_trippers.go:580] Date: Tue, 30 Aug 2022 05:54:43 GMT .:53 [INFO] plugin/reload: Running configuration SHA512 = 53330204b5b8aeef5f056d933359bc7e95d66a7fa922f7eef2e4e80e0962233039fd776606db297a43f0847fef9243d143e67037cf901385bcb310c27712d2c1 CoreDNS-1.9.3 linux/arm64, go1.18.2, 45b0a11 error: stream error: stream ID 3; INTERNAL_ERROR; received from peer Tue Aug 30 06:04:35 UTC 2022 ``` Two following runs also show the error after 9-10 minutes.
issue