Gitlab::HttpIO::FailedToGetChunkError: Gitlab::HttpIO::FailedToGetChunkError

https://sentry.gitlab.net/gitlab/gitlabcom/issues/934146/

Gitlab::HttpIO::FailedToGetChunkError: Gitlab::HttpIO::FailedToGetChunkError
  gitlab/http_io.rb:156:in `get_chunk'
    raise FailedToGetChunkError unless response.code == '200' || response.code == '206'
  gitlab/http_io.rb:84:in `read'
    data = get_chunk
  net/http/generic_request.rb:207:in `copy_stream'
    IO.copy_stream(f, sock.io)
  net/http/generic_request.rb:207:in `send_request_with_body_stream'
    IO.copy_stream(f, sock.io)
  net/http/generic_request.rb:125:in `exec'
    send_request_with_body_stream sock, ver, path, @body_stream
...
(75 additional frame(s) were not displayed)

Gitlab::HttpIO::FailedToGetChunkError: Gitlab::HttpIO::FailedToGetChunkError

Solution

As described in the comment #31744 (comment 975715114), this error occurs because the upload to the external source takes too long to complete and the URL of the source file in the object-store expires.

So the proposed solution is to make the upload quicker by not using Gitlab::HttpIO#read that makes several calls to the object store requesting a small chunk of the file, and instead, read the file as a stream. See this comment for an example of the code