Skip to content

Geo: Fix redirects of LFS transfer downloads

Michael Kozono requested to merge mk/fix-lfs-transfer-redirection into master

What does this MR do and why?

Describe in detail what your merge request does and why.

Geo: Fix redirects of LFS transfer downloads

If an LFS transfer download is redirected, then the Git LFS client does not know how to reauthenticate with the new remote (the primary Geo site). So those LFS transfer download requests which need authentication would fail.

If the initial LFS batch download request succeeded without redirect, then we expect the secondary has all the relevant LFS objects locally already. So it's safe to not redirect any LFS transfer download requests.

Resolves #371571 (closed)

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

  1. Set up GitLab with a Geo secondary site with a different external URL than the primary
  2. Set up a project with LFS files
  3. Clone the project from the secondary
  4. In GitLab UI, add to the main branch 2 large LFS files which would each take more than 2 seconds to download (say 300MB)
  5. Set up a script to push a small new change to master in a loop, for example: 
    loop_git_push_example_txt() { duration=$1; echo "Loop for $duration seconds"; timeout $duration bash -c 'while (echo "Bump" >> example.txt && git commit -am "Bump example.txt" && git push); do sleep 0.1; done'; say 'Finished many Git pushes' }
  6. Run the loop script
  7. git pull in the local repo

Before:

[2022-08-22T05:36:03.096Z] 05:36:03.051717 trace git-lfs: creds: git credential fill ("https", "gitlab.primary.example.org", "")
[2022-08-22T05:36:03.096Z] 05:36:03.053318 git.c:439               trace: built-in: git credential fill
[2022-08-22T05:36:03.096Z] fatal: could not read Username for 'gitlab.primary.example.org': No such device or address
[2022-08-22T05:36:03.096Z] 05:36:03.053622 trace git-lfs: api error: Git credentials for https://gitlab.primary.example.org/-/push_from_secondary/5/group/project.git/info/lfs/objects/batch not found.

After: Git pull succeeds and downloads both LFS files

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Ian Baum

Merge request reports