Resolve "Proxied LFS Object requests are being redirected to the primary's external URL"

Natanael Silvarequested to merge
543956-fix-lfs-download-redirection into master

What does this MR do and why?

It fixes an issue for failed downloads of LFS objects when geo was configured with unified URL and the request was redirected due to the local repo being out-of-date.

The download links built by the requests to POST /info/lfs/objects/batch were using the primary's URL (which is the same as the unified URL). When the client sends those download requests to the Unified URL, if the load balancer/DNS sends those to the secondary node (depending on the routing rules), then the secondary node could receive download requests for objects that it does not have yet.

References

Request for Help that triggered the issue being fixed.

Sequence Diagrams

Before (Broken Flow)

sequenceDiagram
    participant Client as Git LFS Client
    participant LB as Load Balancer/DNS (at Unified URL)
    participant Secondary as Secondary Node
    participant Primary as Primary Node

    Note over Client, Primary: LFS Object Download Request (Unified URL Setup)
    
    Client->>LB: POST /info/lfs/objects/batch
    LB->>Secondary: POST /info/lfs/objects/batch
    
    Note over Secondary: Project not synced locally
    Secondary->>Secondary: Redirect to primary needed
    Secondary-->>LB: 302 Redirect to /-/from_secondary/2/project/info/lfs/objects/batch
    LB-->>Client: 302 Redirect
    
    Client->>LB: POST /-/from_secondary/2/project/info/lfs/objects/batch
    LB->>Secondary: POST /-/from_secondary/2/project/info/lfs/objects/batch
    Secondary->>Primary: Forward request to primary

    Primary-->>Secondary: Respond with download URLs like http://unified-url/project/gitlab-lfs/objects/oid
    
    Secondary-->>LB: Forward response
    LB-->>Client: Forward response
    
    Note over Client: Client tries to download from Unified URL
    Client->>LB: GET http://unified-url/project/gitlab-lfs/objects/oid
    
    Note over LB: ❌ Load balancer/DNS may always route this user to ❌<br />❌ the secondary depending on its routing rules  ❌ 
    
    LB->>Secondary: GET /project/gitlab-lfs/objects/oid
    
    Note over Secondary: ❌ Object not available locally. Secondary doesn't know❌<br />❌how to accelerate individual LFS oid download requests  ❌ 
    Secondary-->>LB: 404 Not found
    LB-->>Client: 404 Not found
    
    Note over Client: ❌ Download fails ❌

After (Fixed Flow)

sequenceDiagram
    participant Client as Git LFS Client
    participant LB as Load Balancer/DNS (at Unified URL)
    participant Secondary as Secondary Node
    participant Primary as Primary Node

    Note over Client, Primary: LFS Object Download Request (Unified URL Setup - Fixed)
    
    Client->>LB: POST /info/lfs/objects/batch
    LB->>Secondary: POST /info/lfs/objects/batch
    
    Note over Secondary: Project not synced locally
    Secondary->>Secondary: Redirect to primary needed
    Secondary-->>LB: 302 Redirect to /-/from_secondary/2/project/info/lfs/objects/batch
    LB-->>Client: 302 Redirect
    
    Client->>LB: POST /-/from_secondary/2/project/info/lfs/objects/batch
    LB->>Secondary: POST /-/from_secondary/2/project/info/lfs/objects/batch
    Secondary->>Primary: Forward request to primary
    Note over Primary: No change up to this point
    
    Note over Primary: ✅ FIX: Return URL with `/-/from_secondary` prefix ✅

    Primary-->>Secondary: Respond with download URLs like http://unified-url/-/from_secondary/project/gitlab-lfs/objects/oid
    
    Secondary-->>LB: Forward response
    LB-->>Client: Forward response
    
    Note over Client: Client requests download URLs
    Client->>LB: GET http://unified-url/-/from_secondary/project/gitlab-lfs/objects/oid
    LB->>Secondary: Forward to secondary according to routing rules

    Note over Secondary: ✅ Always proxy `/-/from_secondary` paths to primary ✅
    Secondary->>Primary: Proxy to primary
    Primary-->>Secondary: 200 OK (send the LFS object file or S3-signed URL)
    Secondary-->>LB: 200 OK
    LB-->Client: 200 OK

Screenshots or screen recordings

Cloning a repository with git lfs enabled, and no redirection

Click to expand Git trace 
dev|⇒ rm -Rf ./nonsynced && GIT_LFS_DEBUG=1 GIT_LFS_TRACE=1 GIT_TRACE=1 git clone http://nonhacker:farofa88@127.0.0.1:3000/grupao/nonsynced.git
16:58:23.664348 git.c:476               trace: built-in: git clone http://nonhacker:farofa88@127.0.0.1:3000/grupao/nonsynced.git
Cloning into 'nonsynced'...
16:58:23.672909 run-command.c:667       trace: run_command: git remote-http origin http://nonhacker:farofa88@127.0.0.1:3000/grupao/nonsynced.git
16:58:23.672923 run-command.c:759       trace: start_command: /opt/homebrew/opt/git/libexec/git-core/git remote-http origin http://nonhacker:farofa88@127.0.0.1:3000/grupao/nonsynced.git
16:58:23.682526 git.c:772               trace: exec: git-remote-http origin http://nonhacker:farofa88@127.0.0.1:3000/grupao/nonsynced.git
16:58:23.682858 run-command.c:667       trace: run_command: git-remote-http origin http://nonhacker:farofa88@127.0.0.1:3000/grupao/nonsynced.git
16:58:23.683067 run-command.c:759       trace: start_command: /opt/homebrew/opt/git/libexec/git-core/git-remote-http origin http://nonhacker:farofa88@127.0.0.1:3000/grupao/nonsynced.git
16:58:24.230265 run-command.c:667       trace: run_command: 'git credential-osxkeychain store'
16:58:24.230443 run-command.c:759       trace: start_command: /bin/sh -c 'git credential-osxkeychain store' 'git credential-osxkeychain store'
16:58:24.251788 git.c:772               trace: exec: git-credential-osxkeychain store
16:58:24.252093 run-command.c:667       trace: run_command: git-credential-osxkeychain store
16:58:24.252282 run-command.c:759       trace: start_command: /opt/homebrew/opt/git/libexec/git-core/git-credential-osxkeychain store
remote: Enumerating objects: 17, done.
16:58:24.506845 run-command.c:667       trace: run_command: git index-pack --stdin -v --fix-thin '--keep=fetch-pack 98474 on nsilva--20240819-VQ546' --check-self-contained-and-connected
remote: Counting objects: 100% (2/2), done.
remote: Compressing objects: 100% (2/2), done.
16:58:24.506884 run-command.c:759       trace: start_command: /opt/homebrew/opt/git/libexec/git-core/git index-pack --stdin -v --fix-thin '--keep=fetch-pack 98474 on nsilva--20240819-VQ546' --check-self-contained-and-connected
remote: Total 17 (delta 0), reused 0 (delta 0), pack-reused 15 (from 1)
16:58:24.514452 git.c:476               trace: built-in: git index-pack --stdin -v --fix-thin '--keep=fetch-pack 98474 on nsilva--20240819-VQ546' --check-self-contained-and-connected
Receiving objects: 100% (17/17), 4.20 KiB | 4.20 MiB/s, done.
Resolving deltas: 100% (1/1), done.
16:58:24.517808 run-command.c:667       trace: run_command: git rev-list --objects --stdin --not --all --quiet --alternate-refs '--progress=Checking connectivity'
16:58:24.517829 run-command.c:759       trace: start_command: /opt/homebrew/opt/git/libexec/git-core/git rev-list --objects --stdin --not --all --quiet --alternate-refs '--progress=Checking connectivity'
16:58:24.525104 git.c:476               trace: built-in: git rev-list --objects --stdin --not --all --quiet --alternate-refs '--progress=Checking connectivity'
16:58:24.529136 run-command.c:667       trace: run_command: 'git-lfs filter-process'
16:58:24.529141 run-command.c:759       trace: start_command: /bin/sh -c 'git-lfs filter-process' 'git-lfs filter-process'
16:58:24.602332 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' '--git-dir' '--show-toplevel'
16:58:24.612334 trace git-lfs: exec: git 'rev-parse' '--is-bare-repository'
16:58:24.619926 trace git-lfs: exec: git 'config' '--includes' '--local' 'lfs.repositoryformatversion'
16:58:24.626668 trace git-lfs: exec: git 'config' '--includes' '--replace-all' 'lfs.repositoryformatversion' '0'
16:58:24.636505 trace git-lfs: exec: git 'config' '--includes' '-l'
16:58:24.643155 trace git-lfs: exec: git 'rev-parse' '--is-bare-repository'
16:58:24.650219 trace git-lfs: exec: git 'config' '--includes' '-l' '--blob' ':.lfsconfig'
16:58:24.656996 trace git-lfs: exec: git 'config' '--includes' '-l' '--blob' 'HEAD:.lfsconfig'
16:58:24.664683 trace git-lfs: Install hook: pre-push, force=false, path=/Users/natanaelsilva/dev/nonsynced/.git/hooks/pre-push
16:58:24.664862 trace git-lfs: Install hook: post-checkout, force=false, path=/Users/natanaelsilva/dev/nonsynced/.git/hooks/post-checkout
16:58:24.664949 trace git-lfs: Install hook: post-commit, force=false, path=/Users/natanaelsilva/dev/nonsynced/.git/hooks/post-commit
16:58:24.665032 trace git-lfs: Install hook: post-merge, force=false, path=/Users/natanaelsilva/dev/nonsynced/.git/hooks/post-merge
16:58:24.665098 trace git-lfs: Initialize filter-process
16:58:24.665227 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' 'HEAD' '--symbolic-full-name' 'HEAD'
16:58:24.671668 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' '--git-dir'
16:58:24.679692 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'remote'
16:58:24.688431 trace git-lfs: tq: running as batched queue, batch size of 100
16:58:24.689005 trace git-lfs: filepathfilter: accepting "some_test.txt"
16:58:24.689542 trace git-lfs: tq: sending batch of size 1
16:58:24.690599 trace git-lfs: api: batch 1 files
16:58:24.690724 trace git-lfs: exec: /usr/bin/security 'list-keychains'
16:58:24.706679 trace git-lfs: exec: /usr/bin/security 'find-certificate' '-a' '-p' '-c' '127.0.0.1:3000' '/Library/Keychains/System.keychain'
16:58:24.722392 trace git-lfs: exec: /usr/bin/security 'find-certificate' '-a' '-p' '-c' '127.0.0.1' '/Library/Keychains/System.keychain'
16:58:24.738627 trace git-lfs: HTTP: POST http://nonhacker:farofa88@127.0.0.1:3000/grupao/nonsynced.git/info/lfs/objects/batch
16:58:25.646452 trace git-lfs: HTTP: 200
16:58:25.646679 trace git-lfs: HTTP: {"objects":[{"oid":"f3f304a49cb3eab29976fca4bd18865ae57c24fa3dd47489454e51ec15fc0bf5","size":11,"actions":{"download":{"href":"http://127.0.0.1:3000/grupao/nonsynced.git/gitlab-lfs/objects/f3f304a49cb3eab29976fca4bd18865ae57c24fa3dd47489454e51ec15fc0bf5","header":{"Authorization":"Basic bm9uaGFja2VyOmV5SjBlWEFpT2lKS1YxUWlMQ0poYkdjaU9pSklVekkxTmlKOS5leUprWVhSaElqcDdJbUZqZEc5eUlqb2libTl1YUdGamEyVnlJaXdpWTI5dWRHRnBibVZ5WDJkcFpDSTZJbWRwWkRvdkwyZHBkR3hoWWk5UWNtOXFaV04wTHpJd0luMHNJbXAwYVNJNkltSm1NV0ptTXpCa0xXUmta
16:58:25.646722 trace git-lfs: HTTP: V1l0TkRBeE1pMWlOMkU0TFRnNVpEVXlPRFppWVRZeU5pSXNJbWxoZENJNk1UYzBPRFkwT1RVd05Td2libUptSWpveE56UTROalE1TlRBd0xDSmxlSEFpT2pFM05EZzJOVFkzTURWOS5QWC1vZ2ZCXzZDWVRhNU9oU01QWDh6cE9DWGc3WW91bS1oQmE0UXA0bnZV"}}},"authenticated":true}]}
16:58:25.646803 trace git-lfs: tq: starting transfer adapter "basic"
16:58:25.647469 trace git-lfs: HTTP: GET http://127.0.0.1:3000/grupao/nonsynced.git/gitlab-lfs/objects/f3f304a49cb3eab29976fca4bd18865ae57c24fa3dd47489454e51ec15fc0bf5
16:58:25.865022 trace git-lfs: HTTP: 200
16:58:25.865984 trace git-lfs: filepathfilter: accepting "some_test.txt"
16:58:25.866576 run-command.c:1535      run_processes_parallel: preparing to run up to 1 tasks
16:58:25.866681 run-command.c:667       trace: run_command: /Users/natanaelsilva/dev/nonsynced/.git/hooks/post-checkout 0000000000000000000000000000000000000000 89c7db0c8c31e02a8ee90f8bf7748abce2bb9659 1
16:58:25.866687 run-command.c:759       trace: start_command: /Users/natanaelsilva/dev/nonsynced/.git/hooks/post-checkout 0000000000000000000000000000000000000000 89c7db0c8c31e02a8ee90f8bf7748abce2bb9659 1
16:58:26.158905 git.c:772               trace: exec: git-lfs post-checkout 0000000000000000000000000000000000000000 89c7db0c8c31e02a8ee90f8bf7748abce2bb9659 1
16:58:26.159206 run-command.c:667       trace: run_command: git-lfs post-checkout 0000000000000000000000000000000000000000 89c7db0c8c31e02a8ee90f8bf7748abce2bb9659 1
16:58:26.159248 run-command.c:759       trace: start_command: /opt/homebrew/bin/git-lfs post-checkout 0000000000000000000000000000000000000000 89c7db0c8c31e02a8ee90f8bf7748abce2bb9659 1
16:58:26.189767 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' '--git-dir' '--show-toplevel'
16:58:26.198566 trace git-lfs: exec: git 'config' '--includes' '-l'
16:58:26.206530 trace git-lfs: exec: git 'rev-parse' '--is-bare-repository'
16:58:26.215214 trace git-lfs: exec: git 'config' '--includes' '-l' '--blob' ':.lfsconfig'
16:58:26.222924 trace git-lfs: exec: git 'config' '--includes' '-l' '--blob' 'HEAD:.lfsconfig'
16:58:26.234019 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' 'HEAD' '--symbolic-full-name' 'HEAD'
16:58:26.243801 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' '--git-dir'
16:58:26.251992 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'remote'
16:58:26.260812 trace git-lfs: NewLsFiles: running in /Users/natanaelsilva/dev/nonsynced git ls-files -z --cached --sparse --exclude-standard --others
16:58:26.260836 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'ls-files' '-z' '--cached' '--sparse' '--exclude-standard' '--others'
16:58:26.270746 trace git-lfs: findAttributeFiles: located .gitattributes
16:58:26.272071 run-command.c:1562      run_processes_parallel: done
16:58:26.272154 run-command.c:56        trace: run_command: running exit handler for pid 98490
16:58:26.272314 trace git-lfs: filepathfilter: creating pattern ".git" of type gitignore
16:58:26.272332 trace git-lfs: filepathfilter: creating pattern "**/.git" of type gitignore
16:58:26.272385 trace git-lfs: filepathfilter: accepting "tmp"

Cloning a repository with git lfs enabled, from the secondary, where the project is not synced:

Click to expand Git trace 
dev|⇒ rm -Rf ./nonsynced && GIT_LFS_DEBUG=1 GIT_LFS_TRACE=1 GIT_TRACE=1 git clone http://nonhacker:farofa88@127.0.0.1:3001/grupao/nonsynced.git
17:00:17.015699 git.c:476               trace: built-in: git clone http://nonhacker:farofa88@127.0.0.1:3001/grupao/nonsynced.git
Cloning into 'nonsynced'...
17:00:17.022895 run-command.c:667       trace: run_command: git remote-http origin http://nonhacker:farofa88@127.0.0.1:3001/grupao/nonsynced.git
17:00:17.022909 run-command.c:759       trace: start_command: /opt/homebrew/opt/git/libexec/git-core/git remote-http origin http://nonhacker:farofa88@127.0.0.1:3001/grupao/nonsynced.git
17:00:17.030832 git.c:772               trace: exec: git-remote-http origin http://nonhacker:farofa88@127.0.0.1:3001/grupao/nonsynced.git
17:00:17.031135 run-command.c:667       trace: run_command: git-remote-http origin http://nonhacker:farofa88@127.0.0.1:3001/grupao/nonsynced.git
17:00:17.031157 run-command.c:759       trace: start_command: /opt/homebrew/opt/git/libexec/git-core/git-remote-http origin http://nonhacker:farofa88@127.0.0.1:3001/grupao/nonsynced.git
17:00:17.766663 run-command.c:667       trace: run_command: 'git credential-osxkeychain store'
17:00:17.766876 run-command.c:759       trace: start_command: /bin/sh -c 'git credential-osxkeychain store' 'git credential-osxkeychain store'
17:00:17.788871 git.c:772               trace: exec: git-credential-osxkeychain store
17:00:17.789211 run-command.c:667       trace: run_command: git-credential-osxkeychain store
17:00:17.789228 run-command.c:759       trace: start_command: /opt/homebrew/opt/git/libexec/git-core/git-credential-osxkeychain store
warning: redirecting to http://127.0.0.1:3001/-/from_secondary/2/grupao/nonsynced.git/
remote: Enumerating objects: 17, done.
17:00:18.034315 run-command.c:667       trace: run_command: git index-pack --stdin -v --fix-thin '--keep=fetch-pack 98624 on nsilva--20240819-VQ546' --check-self-contained-and-connected
remote: Counting objects: 100% (2/2), done.
remote: Compressing objects: 100% (2/2), done.
17:00:18.034351 run-command.c:759       trace: start_command: /opt/homebrew/opt/git/libexec/git-core/git index-pack --stdin -v --fix-thin '--keep=fetch-pack 98624 on nsilva--20240819-VQ546' --check-self-contained-and-connected
remote: Total 17 (delta 0), reused 0 (delta 0), pack-reused 15 (from 1)
17:00:18.042883 git.c:476               trace: built-in: git index-pack --stdin -v --fix-thin '--keep=fetch-pack 98624 on nsilva--20240819-VQ546' --check-self-contained-and-connected
Receiving objects: 100% (17/17), 4.20 KiB | 4.20 MiB/s, done.
Resolving deltas: 100% (1/1), done.
17:00:18.045843 run-command.c:667       trace: run_command: git rev-list --objects --stdin --not --all --quiet --alternate-refs '--progress=Checking connectivity'
17:00:18.045856 run-command.c:759       trace: start_command: /opt/homebrew/opt/git/libexec/git-core/git rev-list --objects --stdin --not --all --quiet --alternate-refs '--progress=Checking connectivity'
17:00:18.052881 git.c:476               trace: built-in: git rev-list --objects --stdin --not --all --quiet --alternate-refs '--progress=Checking connectivity'
17:00:18.056620 run-command.c:667       trace: run_command: 'git-lfs filter-process'
17:00:18.056624 run-command.c:759       trace: start_command: /bin/sh -c 'git-lfs filter-process' 'git-lfs filter-process'
17:00:18.127051 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' '--git-dir' '--show-toplevel'
17:00:18.136294 trace git-lfs: exec: git 'rev-parse' '--is-bare-repository'
17:00:18.143851 trace git-lfs: exec: git 'config' '--includes' '--local' 'lfs.repositoryformatversion'
17:00:18.151944 trace git-lfs: exec: git 'config' '--includes' '--replace-all' 'lfs.repositoryformatversion' '0'
17:00:18.159381 trace git-lfs: exec: git 'config' '--includes' '-l'
17:00:18.166511 trace git-lfs: exec: git 'rev-parse' '--is-bare-repository'
17:00:18.173728 trace git-lfs: exec: git 'config' '--includes' '-l' '--blob' ':.lfsconfig'
17:00:18.181076 trace git-lfs: exec: git 'config' '--includes' '-l' '--blob' 'HEAD:.lfsconfig'
17:00:18.189495 trace git-lfs: Install hook: pre-push, force=false, path=/Users/natanaelsilva/dev/nonsynced/.git/hooks/pre-push
17:00:18.189724 trace git-lfs: Install hook: post-checkout, force=false, path=/Users/natanaelsilva/dev/nonsynced/.git/hooks/post-checkout
17:00:18.189860 trace git-lfs: Install hook: post-commit, force=false, path=/Users/natanaelsilva/dev/nonsynced/.git/hooks/post-commit
17:00:18.190026 trace git-lfs: Install hook: post-merge, force=false, path=/Users/natanaelsilva/dev/nonsynced/.git/hooks/post-merge
17:00:18.190176 trace git-lfs: Initialize filter-process
17:00:18.190310 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' 'HEAD' '--symbolic-full-name' 'HEAD'
17:00:18.197742 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' '--git-dir'
17:00:18.204767 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'remote'
17:00:18.213400 trace git-lfs: tq: running as batched queue, batch size of 100
17:00:18.215213 trace git-lfs: filepathfilter: accepting "some_test.txt"
17:00:18.215752 trace git-lfs: tq: sending batch of size 1
17:00:18.217309 trace git-lfs: api: batch 1 files
17:00:18.217902 trace git-lfs: exec: /usr/bin/security 'list-keychains'
17:00:18.236217 trace git-lfs: exec: /usr/bin/security 'find-certificate' '-a' '-p' '-c' '127.0.0.1:3001' '/Library/Keychains/System.keychain'
17:00:18.258627 trace git-lfs: exec: /usr/bin/security 'find-certificate' '-a' '-p' '-c' '127.0.0.1' '/Library/Keychains/System.keychain'
17:00:18.275746 trace git-lfs: HTTP: POST http://nonhacker:farofa88@127.0.0.1:3001/grupao/nonsynced.git/info/lfs/objects/batch
17:00:18.361898 trace git-lfs: http: decompressed gzipped response
17:00:18.361910 trace git-lfs: HTTP: 302
17:00:18.361917 trace git-lfs: api: redirect POST http://nonhacker:farofa88@127.0.0.1:3001/grupao/nonsynced.git/info/lfs/objects/batch to http://127.0.0.1:3001/-/from_secondary/2/grupao/nonsynced.git/info/lfs/objects/batch
17:00:18.361991 trace git-lfs: HTTP: POST http://127.0.0.1:3001/-/from_secondary/2/grupao/nonsynced.git/info/lfs/objects/batch
17:00:18.550745 trace git-lfs: HTTP: 200
17:00:18.550839 trace git-lfs: HTTP: {"objects":[{"oid":"f3f304a49cb3eab29976fca4bd18865ae57c24fa3dd47489454e51ec15fc0bf5","size":11,"actions":{"download":{"href":"http://127.0.0.1:3000/-/from_secondary/2/grupao/nonsynced.git/gitlab-lfs/objects/f3f304a49cb3eab29976fca4bd18865ae57c24fa3dd47489454e51ec15fc0bf5","header":{}}},"authenticated":true}]}
17:00:18.550979 trace git-lfs: tq: starting transfer adapter "basic"
17:00:18.551651 trace git-lfs: exec: /usr/bin/security 'list-keychains'
17:00:18.571022 trace git-lfs: exec: /usr/bin/security 'find-certificate' '-a' '-p' '-c' '127.0.0.1:3000' '/Library/Keychains/System.keychain'
17:00:18.587280 trace git-lfs: exec: /usr/bin/security 'find-certificate' '-a' '-p' '-c' '127.0.0.1' '/Library/Keychains/System.keychain'
17:00:18.604214 trace git-lfs: HTTP: GET http://127.0.0.1:3000/-/from_secondary/2/grupao/nonsynced.git/gitlab-lfs/objects/f3f304a49cb3eab29976fca4bd18865ae57c24fa3dd47489454e51ec15fc0bf5
17:00:18.676759 trace git-lfs: HTTP: 200
17:00:18.677241 trace git-lfs: filepathfilter: accepting "some_test.txt"
17:00:18.677714 run-command.c:1535      run_processes_parallel: preparing to run up to 1 tasks
17:00:18.677735 run-command.c:667       trace: run_command: /Users/natanaelsilva/dev/nonsynced/.git/hooks/post-checkout 0000000000000000000000000000000000000000 89c7db0c8c31e02a8ee90f8bf7748abce2bb9659 1
17:00:18.677739 run-command.c:759       trace: start_command: /Users/natanaelsilva/dev/nonsynced/.git/hooks/post-checkout 0000000000000000000000000000000000000000 89c7db0c8c31e02a8ee90f8bf7748abce2bb9659 1
17:00:18.833856 git.c:772               trace: exec: git-lfs post-checkout 0000000000000000000000000000000000000000 89c7db0c8c31e02a8ee90f8bf7748abce2bb9659 1
17:00:18.834200 run-command.c:667       trace: run_command: git-lfs post-checkout 0000000000000000000000000000000000000000 89c7db0c8c31e02a8ee90f8bf7748abce2bb9659 1
17:00:18.834245 run-command.c:759       trace: start_command: /opt/homebrew/bin/git-lfs post-checkout 0000000000000000000000000000000000000000 89c7db0c8c31e02a8ee90f8bf7748abce2bb9659 1
17:00:18.871334 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' '--git-dir' '--show-toplevel'
17:00:18.881332 trace git-lfs: exec: git 'config' '--includes' '-l'
17:00:18.890547 trace git-lfs: exec: git 'rev-parse' '--is-bare-repository'
17:00:18.899755 trace git-lfs: exec: git 'config' '--includes' '-l' '--blob' ':.lfsconfig'
17:00:18.909559 trace git-lfs: exec: git 'config' '--includes' '-l' '--blob' 'HEAD:.lfsconfig'
17:00:18.920627 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' 'HEAD' '--symbolic-full-name' 'HEAD'
17:00:18.930584 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' '--git-dir'
17:00:18.939164 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'remote'
17:00:18.949857 trace git-lfs: NewLsFiles: running in /Users/natanaelsilva/dev/nonsynced git ls-files -z --cached --sparse --exclude-standard --others
17:00:18.949882 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'ls-files' '-z' '--cached' '--sparse' '--exclude-standard' '--others'
17:00:18.959707 trace git-lfs: findAttributeFiles: located .gitattributes
17:00:18.961257 run-command.c:1562      run_processes_parallel: done
17:00:18.961369 run-command.c:56        trace: run_command: running exit handler for pid 98642
17:00:18.961580 trace git-lfs: filepathfilter: creating pattern ".git" of type gitignore
17:00:18.961606 trace git-lfs: filepathfilter: creating pattern "**/.git" of type gitignore
17:00:18.961689 trace git-lfs: filepathfilter: accepting "tmp"

How to set up and validate locally

  1. Set up selective sync on a secondary

  2. Create a project in a group that is not synced to the secondary

  3. clone the project from the secondary, that will force it to redirect the request to the primary

  4. Notice the individual object download requests (something like gitlab-lfs/objects/f3f304a49cb3eab29976fca4bd18865ae57c24fa3dd47489454e51ec15fc0bf5

    should also have "from_secondary/{node_id}".

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #543956 (closed)

Edited by Michael Kozono

Merge request reports