Skip to content

Do not buffer authentication requests in Nginx

Request body (such as git push) is not consumed by gitlab-workhorse when we determine that client did not provide valid credentials. If we allow Nginx to buffer responses in this situation, it may sometimes close the connection without sending the complete response.

This in turns interrupts the authentication process, while certain authentication protocols like Kerberos require several roundtrips. Intermittent issues with git push using Kerberos authentication is what led us to identify this issue. Presumably there are some conditions where Nginx finds it a problem to still have buffered request data when the response is sent, and drops the connection. When it happens the following can be seen in the nginx error log:

2016/12/05 18:05:31 [error] 21540#0: *3422222 readv() failed (104: Connection reset by peer) while reading upstream

An alternative solution is to consume the request body with something like ioutil.ReadAll(r.Body) but it does not seem justified to force the client to send the complete body when we know we won't do anything with it. This problem seems specific to Nginx when buffering both requests and responses; other proxies (especially ones not buffering the request) would typically not be affected so it seems to make sense to disable the problematic behavior specific to Nginx.

Merge request reports