Git LFS auth issue when using Runner with Docker executor DIND

Summary

I cannot run jobs that have Git LFS objects when the executor is docker-in-docker.

Steps to reproduce

Following the instructions here: https://docs.gitlab.com/ee/workflow/lfs/manage_large_binaries_with_git_lfs.html

$ git lfs install
$ git lfs track "*.zip"
$ git add filename.zip
$ git commit -m "Added zip file"
$ git push

This works so far. However, the Runner fails to build the job. This is what's in the .gitlab-ci.yml:

image: node:8-alpine
services:
  - docker:dind

What is the current bug behavior?

Runner fails to execute job with LFS objects.

What is the expected correct behavior?

Runner should be able to run jobs with LFS objects.

Relevant logs and/or screenshots

Running with gitlab-runner 10.7.2 (b5e03c94)
  on reston-hq e015ee9d
Using Docker executor with image node:8-alpine ...
Starting service docker:dind ...
Pulling docker image docker:dind ...
Using docker image sha256:1f44348b3ad523d5dc4ae7d53bd873879e06e0df2d686e9029a666945443ef42 for docker:dind ...
Waiting for services to be up and running...
Pulling docker image node:8-alpine ...
Using docker image sha256:fc3b0429ffb5f43dc473256ef228c87930c637847faad2b5f4e8a0d287111cf2 for node:8-alpine ...
Running on runner-e015ee9d-project-107-concurrent-0 via build...
Fetching changes...
Removing .gitattributes
Removing .gitignore
Removing README.md
Removing dist/
Removing docs/
Downloading filename.zip (451 MB)
warning: current Git remote contains credentials
Error downloading object: filename.zip (8f5510a): Smudge error: Error downloading filename.zip (8f5510ac38947a8e53f0dc2c446eadc58f4a9dc03da4bc26ef08a763aa8102ea): batch response: Authentication required: Authorization error: https://gitlab-ci-token:REDACTED-TOKEN@git.domain.com/org/project.git/info/lfs/objects/batch
Check that you have proper access to the repository

Errors logged to /builds/org/project/.git/lfs/objects/logs/20180517T054632.062001077.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: filename.zip: smudge filter lfs failed
ERROR: Job failed: exit code 1

Results of GitLab environment info

Expand for output related to GitLab environment info

System information
System:		Ubuntu 16.04
Proxy:		no
Current User:	git
Using RVM:	no
Ruby Version:	2.3.6p384
Gem Version:	2.6.13
Bundler Version:1.13.7
Rake Version:	12.3.0
Redis Version:	3.2.11
Git Version:	2.14.3
Sidekiq Version:5.0.5
Go Version:	unknown
GitLab information
Version:	10.7.3-ee
Revision:	584a495
Directory:	/opt/gitlab/embedded/service/gitlab-rails
DB Adapter:	postgresql
DB Version:	9.6.8
URL:		https://git.patrocinium.com
HTTP Clone URL:	https://git.patrocinium.com/some-group/some-project.git
SSH Clone URL:	git@git.patrocinium.com:some-group/some-project.git
Elasticsearch:	no
Geo:		no
Using LDAP:	no
Using Omniauth:	no
GitLab Shell
Version:	7.1.2
Repository storage paths:

• default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check

Checking GitLab Shell ...
GitLab Shell version >= 7.1.2 ? ... OK (7.1.2)
Repo base directory exists?
default... yes
Repo storage directories are symlinks?
default... no
Repo paths owned by git:root, or git:git?
default... yes
Repo paths access is drwxrws---?
default... yes
hooks directories in repos are links: ...
2/1 ... ok
2/2 ... ok
2/3 ... ok
2/4 ... ok
2/5 ... ok
4/6 ... ok
2/7 ... ok
2/8 ... ok
2/9 ... ok
2/10 ... ok
25/11 ... ok
25/12 ... ok
2/13 ... ok
4/14 ... ok
2/15 ... ok
4/16 ... ok
2/17 ... ok
20/18 ... ok
4/19 ... ok
2/20 ... ok
4/21 ... ok
2/22 ... ok
2/23 ... ok
2/24 ... ok
2/25 ... ok
2/26 ... ok
2/27 ... ok
2/28 ... ok
2/29 ... ok
4/30 ... ok
2/31 ... ok
4/32 ... ok
4/33 ... ok
2/34 ... ok
2/35 ... repository is empty
2/36 ... ok
2/37 ... ok
2/38 ... repository is empty
4/39 ... ok
2/40 ... ok
4/44 ... ok
4/46 ... repository is empty
22/48 ... ok
4/49 ... ok
4/50 ... ok
26/51 ... ok
26/52 ... ok
20/54 ... ok
4/55 ... ok
22/56 ... ok
4/57 ... ok
2/58 ... ok
22/59 ... ok
4/60 ... repository is empty
26/61 ... ok
20/62 ... ok
4/63 ... ok
26/64 ... ok
26/65 ... ok
4/66 ... ok
4/68 ... ok
36/70 ... ok
2/71 ... ok
4/74 ... ok
20/75 ... repository is empty
4/76 ... repository is empty
4/77 ... ok
4/78 ... ok
4/79 ... ok
4/80 ... ok
4/81 ... ok
4/83 ... ok
26/84 ... ok
4/85 ... ok
22/86 ... ok
4/87 ... ok
22/88 ... ok
4/89 ... ok
38/90 ... repository is empty
4/91 ... ok
4/92 ... ok
22/93 ... ok
26/94 ... ok
4/95 ... ok
22/96 ... ok
4/97 ... ok
4/98 ... ok
4/99 ... ok
38/100 ... ok
22/101 ... ok
22/102 ... ok
26/104 ... ok
4/105 ... ok
4/106 ... repository is empty
4/107 ... ok
4/108 ... ok
38/110 ... ok
36/111 ... ok
4/113 ... ok
4/115 ... repository is empty
4/116 ... ok
26/117 ... ok
22/118 ... ok
4/119 ... ok
22/120 ... ok
4/121 ... ok
4/124 ... ok
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Check GitLab API access: OK
Redis available via internal API: OK
Access to /var/opt/gitlab/.ssh/authorized_keys: OK
gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Sidekiq ...
Running? ... yes
Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Reply by email is disabled in config/gitlab.yml
Checking LDAP ...
LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab ...
Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
2/1 ... yes
2/2 ... yes
2/3 ... yes
2/4 ... yes
2/5 ... yes
4/6 ... yes
2/7 ... yes
2/8 ... yes
2/9 ... yes
2/10 ... yes
25/11 ... yes
25/12 ... yes
2/13 ... yes
4/14 ... yes
2/15 ... yes
4/16 ... yes
2/17 ... yes
20/18 ... yes
4/19 ... yes
2/20 ... yes
4/21 ... yes
2/22 ... yes
2/23 ... yes
2/24 ... yes
2/25 ... yes
2/26 ... yes
2/27 ... yes
2/28 ... yes
2/29 ... yes
4/30 ... yes
2/31 ... yes
4/32 ... yes
4/33 ... yes
2/34 ... yes
2/35 ... yes
2/36 ... yes
2/37 ... yes
2/38 ... yes
4/39 ... yes
2/40 ... yes
4/44 ... yes
4/46 ... yes
22/48 ... yes
4/49 ... yes
4/50 ... yes
26/51 ... yes
26/52 ... yes
20/54 ... yes
4/55 ... yes
22/56 ... yes
4/57 ... yes
2/58 ... yes
22/59 ... yes
4/60 ... yes
26/61 ... yes
20/62 ... yes
4/63 ... yes
26/64 ... yes
26/65 ... yes
4/66 ... yes
4/68 ... yes
36/70 ... yes
2/71 ... yes
4/74 ... yes
20/75 ... yes
4/76 ... yes
4/77 ... yes
4/78 ... yes
4/79 ... yes
4/80 ... yes
4/81 ... yes
4/83 ... yes
26/84 ... yes
4/85 ... yes
22/86 ... yes
4/87 ... yes
22/88 ... yes
4/89 ... yes
38/90 ... yes
4/91 ... yes
4/92 ... yes
22/93 ... yes
26/94 ... yes
4/95 ... yes
22/96 ... yes
4/97 ... yes
4/98 ... yes
4/99 ... yes
38/100 ... yes
22/101 ... yes
22/102 ... yes
26/104 ... yes
4/105 ... yes
4/106 ... yes
4/107 ... yes
4/108 ... yes
38/110 ... yes
36/111 ... yes
4/113 ... yes
4/115 ... yes
4/116 ... yes
26/117 ... yes
22/118 ... yes
4/119 ... yes
22/120 ... yes
4/121 ... yes
4/124 ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.3.5 ? ... yes (2.3.6)
Git version >= 2.9.5 ? ... yes (2.14.3)
Git user has default SSH configuration? ... no
Try fixing it:
mkdir ~/gitlab-check-backup-1526537935
sudo mv /var/opt/gitlab/.ssh/id_rsa ~/gitlab-check-backup-1526537935
sudo mv /var/opt/gitlab/.ssh/id_rsa.pub ~/gitlab-check-backup-1526537935
For more information see:
doc/ssh/README.md in section "SSH on the GitLab server"
Please fix the error above and rerun the checks.
Active users: ... 13
Elasticsearch version 5.1 - 5.5? ... skipped (elasticsearch is disabled)
Checking GitLab ... Finished

Possible fixes

I believe this is related to this issue: gitlab-runner#2300