Default Gitaly gRPC Keepalive Policy
Relates to https://gitlab.com/gitlab-org/gitlab-ce/issues/62306
It is possible that during long running gRPC sessions a server can fail "silently" where the client is not notified of the server failure. Normally, a server will try to ping the client and send a final GOAWAY
message to indicate the connection is closing, but their delivery is not guaranteed in a real world environment.
gRPC has a client keep alive feature that allows idle connections to detect when the connection has been lost within a reasonable amount of time. The keep alive policy can be configured to ping the server every so often and determine when it is appropriate to consider a connection as "dead" so that a redial can be attempted.
The keep alive policy can be implemented as part of the default Gitaly client: https://gitlab.com/gitlab-org/gitaly/blob/master/client/dial.go
We may want to also set a server-side enforcement policy to protect the Gitaly server from being abused from excessive pings: https://godoc.org/google.golang.org/grpc#KeepaliveEnforcementPolicy
See the following gRPC document for more details: https://github.com/grpc/grpc-go/blob/master/Documentation/keepalive.md