Improve LFS client performance and fix compatibility with Azure DevOps

What does this MR do and why?

The LFS client that is used in repository mirroring had a few shortcomings:

  1. Chunked transfers were never used because the Content-Length was always specified. We now only including Content-Length if chunked encodings are not requested.

  2. Azure DevOps passes in the username in the upload URL, but this cause HTTParty to serialize this in the Authorization header instead of using the mirror credentials. We now strip the URL of username/passwords if HTTP Basic Auth is used.

Relates to #340482 (closed)

How to set up and validate locally

  1. Set up a Azure DevOps account in dev.azure.com (https://andrewlock.net/creating-a-git-repo-with-azure-repos-and-trying-out-git-lfs/).
  2. Create a project with LFS files.
  3. Set up push mirroring to an Azure mirror.
  4. Notice LFS files never show up, and no error is shown on project settings. exceptions_json.log does have an error.

Or on the console:

project = Project.last
user = User.last
url = 'YOUR-PROJECT-URL'
credentials = { auth_method: 'password', user: 'GIT-CLONE-USER', password: 'GIT-CLONE-PASSWORD' }
Lfs::PushService.new(project, user, url: url, credentials: credentials).execute

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Stan Hu

Merge request reports

Loading