Rails fails to finalize 5+ GB LFS files before Workhorse times out
ZD: https://gitlab.zendesk.com/agent/tickets/77637
Customer has an LFS share mounted via NFS. Pushing files < 5 GB appears to work, although a bit slow. We observed that in the Workhorse logs, after the entire transfer finishes, there is a timeout:
This timeout occurs when Workhorse proxies the request to Rails to finalize the upload: https://gitlab.com/gitlab-org/gitlab-ce/blob/v9.2.2/app/controllers/projects/lfs_storage_controller.rb#L22-32
The Rails log shows that this endpoint is being hit after the transfer finishes. We suspect that it's taking > 10 s for Rails to move the file to the final location, and Workhorse gives up.
Couple of questions:
- Why is the move so slow? We suspect the file is being copied, not being moved, and it's taking more than 10 s.
- Should we increase the timeout on Workhorse for this?
/cc: @cpallares, @nick.thomas, @MikeWalsh