LFS files are not pulled automatically for build jobs

Summary

gitlab-runner does not fetch and checkout LFS files automatically.

Steps to reproduce

Push an LFS file foo that contains I'm an LFS file and a .gitlab-ci.yml like below:

.gitlab-ci.yml

```yml before_script: - type foo ```

Actual behavior

foo is an LFS pointer file containing text like:

version https://git-lfs.github.com/spec/v1
oid sha256:1296330fcf1e09e6c2f926301916f64d4a4c5c0ff12d460a9bc5d4c48411518f
size 123

Expected behavior

The job log should show I'm an LFS file

Relevant logs and/or screenshots

Running on Windows 10 with Portable Git installed. gitlab-runner is launched with the batch script below:

cd %~dp0
set PATH=C:\Git-2.23.0\cmd;%PATH%
start /belownormal gitlab-runner run

Relevant parts of the job log

job log

Running with gitlab-runner 12.3.0 (a8a019e0)
  on lab-ci 34625f0b
Using Shell executor...

C:\glr>setlocal enableextensions 

C:\glr>setlocal enableDelayedExpansion 

C:\glr>set nl=
 

C:\glr>echo Running on LAB-CI... 
Running on LAB-CI...

C:\glr>setlocal enableextensions 

C:\glr>setlocal enableDelayedExpansion 

C:\glr>set nl=
 

C:\glr>SET FF_CMD_DISABLE_DELAYED_ERROR_LEVEL_EXPANSION=false 

C:\glr>SET FF_USE_LEGACY_BUILDS_DIR_FOR_DOCKER=false 

C:\glr>SET FF_USE_LEGACY_VOLUMES_MOUNTING_ORDER=false 

C:\glr>SET CI_RUNNER_SHORT_TOKEN=34625f0b 

C:\glr>SET CI_BUILDS_DIR=C:\glr\builds 

C:\glr>SET CI_PROJECT_DIR=C:\glr\builds\34625f0b\0\sw\genshi 

C:\glr>SET CI_CONCURRENT_ID=0 

C:\glr>SET CI_CONCURRENT_PROJECT_ID=0 

C:\glr>SET CI_SERVER=yes 

C:\glr>md "C:\\glr\\builds\\34625f0b\\0\\sw\\genshi.tmp"  2>NUL 1>NUL 

C:\glr>SET CI_SERVER_TLS_CA_FILE=C:\glr\builds\34625f0b\0\sw\genshi.tmp\CI_SERVER_TLS_CA_FILE 

C:\glr>SET CI_PIPELINE_ID=10941 

C:\glr>SET CI_JOB_ID=62424 

C:\glr>SET CI_JOB_TOKEN=xxxxxxxxxxxxxxxxxxxx 

C:\glr>SET CI_BUILD_ID=62424 

C:\glr>SET CI_BUILD_TOKEN=xxxxxxxxxxxxxxxxxxxx 

C:\glr>SET CI_REGISTRY_USER=gitlab-ci-token 

C:\glr>SET CI_REGISTRY_PASSWORD=xxxxxxxxxxxxxxxxxxxx 

C:\glr>SET CI_REPOSITORY_URL=https://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx@gitlab/sw/genshi.git 

C:\glr>SET CI=true 

C:\glr>SET GITLAB_CI=true 

C:\glr>SET GITLAB_FEATURES= 

C:\glr>SET CI_SERVER_NAME=GitLab 

C:\glr>SET CI_SERVER_VERSION=11.0.3 

C:\glr>SET CI_SERVER_REVISION=aa62075 

C:\glr>SET CI_JOB_NAME=superbuild_ci 

C:\glr>SET CI_JOB_STAGE=superbuild 

C:\glr>SET CI_COMMIT_SHA=818e57563ca3e9905c19cabdc1107a5373e30b54 

C:\glr>SET CI_COMMIT_REF_NAME=feature/CRAN-3032-GLRVersion 

C:\glr>SET CI_COMMIT_REF_SLUG=feature-cran-3032-glrversion 

C:\glr>SET CI_BUILD_REF=818e57563ca3e9905c19cabdc1107a5373e30b54 

C:\glr>SET CI_BUILD_BEFORE_SHA=35e4887427361edc590dcd17155be7d8f973c1e8 

C:\glr>SET CI_BUILD_REF_NAME=feature/CRAN-3032-GLRVersion 

C:\glr>SET CI_BUILD_REF_SLUG=feature-cran-3032-glrversion 

C:\glr>SET CI_BUILD_NAME=superbuild_ci 

C:\glr>SET CI_BUILD_STAGE=superbuild 

C:\glr>SET CI_PROJECT_ID=5 

C:\glr>SET CI_PROJECT_NAME=genshi 

C:\glr>SET CI_PROJECT_PATH=sw/genshi 

C:\glr>SET CI_PROJECT_PATH_SLUG=sw-genshi 

C:\glr>SET CI_PROJECT_NAMESPACE=sw 

C:\glr>SET CI_PROJECT_URL=https://gitlab/sw/genshi 

C:\glr>SET CI_PROJECT_VISIBILITY=internal 

C:\glr>SET CI_REGISTRY=gitlab:5005 

C:\glr>SET CI_REGISTRY_IMAGE=gitlab:5005/sw/genshi 

C:\glr>SET CI_PIPELINE_IID=9863 

C:\glr>SET CI_CONFIG_PATH=.gitlab-ci.yml 

C:\glr>SET CI_PIPELINE_SOURCE=push 

C:\glr>SET CI_COMMIT_MESSAGE=CI_DEBUG_TRACE: "true"!nl! 

C:\glr>SET CI_COMMIT_TITLE=CI_DEBUG_TRACE: "true" 

C:\glr>SET CI_COMMIT_DESCRIPTION= 

C:\glr>SET CI_RUNNER_ID=25 

C:\glr>SET CI_RUNNER_DESCRIPTION=lab-ci 

C:\glr>SET GET_SOURCES_ATTEMPTS=5 

C:\glr>SET CI_DEBUG_TRACE=true 

C:\glr>SET CI_SHARED_ENVIRONMENT=true 

C:\glr>SET CI_RUNNER_VERSION=12.3.0 

C:\glr>SET CI_RUNNER_REVISION=a8a019e0 

C:\glr>SET CI_RUNNER_EXECUTABLE_ARCH=windows/amd64 

C:\glr>SET GIT_LFS_SKIP_SMUDGE=1 

C:\glr>echo Fetching changes... 
Fetching changes...

C:\glr>dir "C:\glr\builds\34625f0b\0\sw\genshi.tmp\git-template"   2>NUL 1>NUL  || md "C:\glr\builds\34625f0b\0\sw\genshi.tmp\git-template"  2>NUL 1>NUL 

C:\glr>"git" "config" "-f" "C:\glr\builds\34625f0b\0\sw\genshi.tmp\git-template/config" "fetch.recurseSubmodules" "false" 

C:\glr>IF !errorlevel! NEQ 0 exit /b !errorlevel! 

C:\glr>"git" "config" "-f" "C:\glr\builds\34625f0b\0\sw\genshi.tmp\git-template/config" "http.https://gitlab.sslCAInfo" "C:\glr\builds\34625f0b\0\sw\genshi.tmp\CI_SERVER_TLS_CA_FILE" 

C:\glr>IF !errorlevel! NEQ 0 exit /b !errorlevel! 

C:\glr>"git" "init" "C:/glr/builds/34625f0b/0/sw/genshi" "--template" "C:\glr\builds\34625f0b\0\sw\genshi.tmp\git-template" 
Reinitialized existing Git repository in C:/glr/builds/34625f0b/0/sw/genshi/.git/

C:\glr>IF !errorlevel! NEQ 0 exit /b !errorlevel! 

C:\glr>cd /D "C:\glr\builds\34625f0b\0\sw\genshi" 

C:\glr\builds\34625f0b\0\sw\genshi>IF !errorlevel! NEQ 0 exit /b !errorlevel! 

C:\glr\builds\34625f0b\0\sw\genshi>del /f /q ".git\index.lock"  2>NUL 1>NUL 

C:\glr\builds\34625f0b\0\sw\genshi>del /f /q ".git\shallow.lock"  2>NUL 1>NUL 

C:\glr\builds\34625f0b\0\sw\genshi>del /f /q ".git\HEAD.lock"  2>NUL 1>NUL 

C:\glr\builds\34625f0b\0\sw\genshi>del /f /q ".git\hooks\post-checkout"  2>NUL 1>NUL 

C:\glr\builds\34625f0b\0\sw\genshi>"git" "remote" "add" "origin" "https://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx@gitlab/sw/genshi.git"  2>NUL 1>NUL 

C:\glr\builds\34625f0b\0\sw\genshi>IF !errorlevel! EQU 0 (echo Created fresh repository. )  ELSE (
"git" "remote" "set-url" "origin" "https://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx@gitlab/sw/genshi.git"  
 IF !errorlevel! NEQ 0 exit /b !errorlevel! 
) 

C:\glr\builds\34625f0b\0\sw\genshi>"git" "fetch" "origin" "--prune" 
From https://gitlab/sw/genshi
   35e488742..818e57563  feature/CRAN-3032-GLRVersion -> origin/feature/CRAN-3032-GLRVersion

C:\glr\builds\34625f0b\0\sw\genshi>IF !errorlevel! NEQ 0 exit /b !errorlevel! 

C:\glr\builds\34625f0b\0\sw\genshi>echo Checking out 818e5756 as feature/CRAN-3032-GLRVersion... 
Checking out 818e5756 as feature/CRAN-3032-GLRVersion...

C:\glr\builds\34625f0b\0\sw\genshi>"git" "checkout" "-f" "-q" "818e57563ca3e9905c19cabdc1107a5373e30b54" 

C:\glr\builds\34625f0b\0\sw\genshi>IF !errorlevel! NEQ 0 exit /b !errorlevel! 

C:\glr\builds\34625f0b\0\sw\genshi>"git" "clean" "-ffdx" 

C:\glr\builds\34625f0b\0\sw\genshi>IF !errorlevel! NEQ 0 exit /b !errorlevel! 

C:\glr\builds\34625f0b\0\sw\genshi>"git-lfs" "version"  2>NUL 1>NUL 

C:\glr\builds\34625f0b\0\sw\genshi>IF !errorlevel! EQU 0 (
"git" "lfs" "pull"  
 IF !errorlevel! NEQ 0 exit /b !errorlevel!  
 echo.
) 

C:\glr\builds\34625f0b\0\sw\genshi>echo Skipping Git submodules setup 
Skipping Git submodules setup

Adding git config -l to the job script shows:

C:\glr\builds\34625f0b\0\sw\genshi>git config -l 
core.symlinks=true
core.autocrlf=true
core.fscache=false
color.diff=auto
color.status=auto
color.branch=auto
color.interactive=true
help.format=html
http.sslcainfo=/ssl/certs/ca-bundle.crt
diff.astextplain.textconv=astextplain
rebase.autosquash=true
filter.lfs.clean=git-lfs clean -- %f
filter.lfs.smudge=git-lfs smudge -- %f
filter.lfs.process=git-lfs filter-process
filter.lfs.required=true
fetch.recursesubmodules=false
http.https://gitlab.sslcainfo=C:\glr\builds\34625f0b\0\sw\genshi.tmp\CI_SERVER_TLS_CA_FILE
core.repositoryformatversion=0
core.filemode=false
core.bare=false
core.logallrefupdates=true
core.ignorecase=true
remote.origin.url=https://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx@gitlab/sw/genshi.git
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*

Environment description

Windows 10
cmd Shell executor
Portable Git for Windows 2.23.0

config.toml contents

concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "lab-ci"
  url = "https://gitlab/"
  executor = "shell"
  shell = "cmd"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]

Used GitLab Runner version

C:\glr>.\gitlab-runner.exe --version
Version:      12.3.0
Git revision: a8a019e0
Git branch:   12-3-stable
GO version:   go1.8.7
Built:        2019-09-20T08:27:18+0000
OS/Arch:      windows/amd64

Possible fixes

I think this is the problem:

C:\glr\builds\34625f0b\0\sw\genshi>"git-lfs" "version"  2>NUL 1>NUL 

C:\glr\builds\34625f0b\0\sw\genshi>IF !errorlevel! EQU 0 (
"git" "lfs" "pull"  
 IF !errorlevel! NEQ 0 exit /b !errorlevel!  
 echo.
)

The check for existence of Git LFS is flawed: git-lfs is not in the system PATH (and shouldn't need to be). This should be git lfs version (same way git lfs pull is used below)

Edited Sep 26, 2019 by Taylor Braun-Jones