Fix gitlab-sshd CPU spin by upgrading golang.org/x/crypto to v0.54.0
What does this MR do?
Upgrades golang.org/x/crypto from v0.52.0 to v0.54.0 to fix a CPU spin / goroutine leak in gitlab-sshd.
Instances with ClientAliveInterval > 0 could pin multiple CPU cores at 100% and accumulate stuck goroutines. CPU and goroutine profiles showed 175 keepalive goroutines all stuck (state [runnable], spinning) in golang.org/x/crypto/ssh.(*mux).SendRequest, with ~100% of CPU time in runtime.selectnbrecv / runtime.chanrecv / runtime.empty — the signature of a non-blocking select spinning in a tight loop.
Root cause
The drain loop added to mux.SendRequest in x/crypto v0.52.0 did not check the receive's ok flag:
drain:
for {
select {
case <-m.globalResponses:
default:
break drain
}
}When a connection is torn down, mux.loop() calls close(m.globalResponses). Receiving from a closed channel in a select always succeeds immediately, so the case arm is perpetually ready, default is never selected, and the loop spins forever. gitlab-shell's sendKeepAliveMsg calls SendRequest(KeepAliveMsg, true, nil) with wantReply=true, so any keepalive firing as/after its connection dies gets permanently wedged.
Upstream fix
golang/crypto commit 4c4d20b — "ssh: fix spinloop in mux SendRequest drain on closed channel" (CL 783681), first released in v0.53.0. It uses the comma-ok idiom to detect the closed channel and break out of the drain loop.
Related
Ref: https://gitlab.com/gitlab-com/request-for-help/-/work_items/5063