Skip to content

Handle backchannel connection closing idempotently

Sami Hiltunen requested to merge smh-fix-backchannel-hang-on-close into master

Backchannel doesn't currently guard against Close() being called multiple times. This can lead to a hang in gRPC when it closes a broken connection.

Closing a backchannel session shutsdown the backchannel gRPC server. The gRPC server closes its listener, which is the yamux session. Closing the yamux session then ultimately closes the underlying TCP connection. The Close method thus waits for the server to return the error from Serve to ensure all of the resources have been cleaned up before returning.

If a connection becomes unhealthy, gRPC will close it from both the writer goroutine and the keepalive checker. The second one will end up blocking forever on the error return as the backchannel server has already been closed and the first closer read the error from the channel. This commit fixes the issue by running the closing logic only once and returning the same error on all subsequent calls to close.

Closes gitlab#351425 (closed)

Merge request reports