executor docker-windows error: cannot lock ref 'refs/remotes/origin/main': unable to resolve reference 'refs/remotes/origin/main': Not a directory

Summary

OS: Windows
Executor: Windows-docker
Mode: local job execution only (remote works)

Job failing after Created fresh repository. log message with error:

cannot lock ref 'refs/remotes/origin/feature/win-ci': unable to resolve reference 'refs/remotes/origin/feature/win-ci': Not a directory

cannot lock ref 'refs/remotes/origin/main': unable to resolve reference 'refs/remotes/origin/main': Not a directory

Remote pipeline execution triggered by pushing into gitlab repo works fine (tested on same machine). Only local run failing.

Only one repository causing this error. Except the runner failing everything seems to be fine with this repo.

Steps to reproduce

This error show up only with one repository that I am not authorized to share. I have no idea what caused this. Please help me navigate closer to source of this problem.

Using old but freshly cloned repo.

git clone (both https and ssh tested)

Run runner locally.

C:\runner\gitlab-runner.exe exec docker-windows --docker-pull-policy=if-not-present hello

Other arguments tested:

--shell powershell
--docker-image mcr.microsoft.com/dotnet/framework/sdk:4.8
--pre-clone-script "git config --global init.defaultBranch main"

Does not proceed to this step, but for completeness.

.gitlab-ci.yml

hello:
  image: mcr.microsoft.com/dotnet/framework/sdk:4.8
  tags:
    - windows
    - container
  stage: build
  script:
    - echo "hello"

Relevant logs and/or screenshots

job log machine1

Runtime platform                                    arch=amd64 os=windows pid=12788 revision=e0218c92 version=14.3.2
WARNING: You most probably have uncommitted changes.
WARNING: These changes will not be tested.
Running with gitlab-runner 14.3.2 (e0218c92)
Preparing the "docker-windows" executor
Using Docker executor with image REDACTED_TESTED_OTHER_IMAGES_TOO ...
Using locally found image version due to "if-not-present" pull policy
Using docker image REDACTED_TESTED_OTHER_IMAGES_TOO ...
Preparing environment
Running on RUNNER--PROJECT via
DESKTOP-4QJM6I4...
Getting source from Git repository
Fetching changes...
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint:   git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint:   git branch -m <name>
Initialized empty Git repository in C:/builds/project-0/.git/
Created fresh repository.
error: cannot lock ref 'refs/remotes/origin/main': unable to resolve reference 'refs/remotes/origin/main': Not a directory
ERROR: Job failed: exit code 1

FATAL: exit code 1

job log machine2

Runtime platform                                    arch=amd64 os=windows pid=1792 revision=7f7a4bb0 version=13.11.0
WARNING: You most probably have uncommitted changes.
WARNING: These changes will not be tested.
Running with gitlab-runner 13.11.0 (7f7a4bb0)
Preparing the "docker-windows" executor
Using Docker executor with image mcr.microsoft.com/dotnet/framework/sdk:4.8 ...
Using locally found image version due to "if-not-present" pull policy
Using docker image sha256:7bfe56cca6e39c3c2b7690f76bc22405dffef3a07e2cae6f8b8452da3bf7e93e for mcr.microsoft.com/dotnet/framework/sdk:4.8 with digest mcr.microsoft.com/dotnet/framework/sdk@sha256:2a755a749e910a45a91f0fe1a84659e717e3b3cf578c9214e983d80f7445fee6 ...
Preparing environment
Running on RUNNER--PROJECT via
plachta-REDACTED...
Getting source from Git repository
Fetching changes...
Initialized empty Git repository in C:/builds/project-0/.git/
Created fresh repository.
error: cannot lock ref 'refs/remotes/origin/feature/win-ci': unable to resolve reference 'refs/remotes/origin/feature/win-ci': Not a directory
error: cannot lock ref 'refs/remotes/origin/main': unable to resolve reference 'refs/remotes/origin/main': Not a directory
ERROR: Job failed: exit code 1
FATAL: exit code 1

Environment description

Machine 1

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Build with BuildKit (Docker Inc., v0.6.1-docker)
  compose: Docker Compose (Docker Inc., v2.0.0-rc.3)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 1
  Running: 0
  Paused: 0
  Stopped: 1
 Images: 253
 Server Version: 20.10.8
 Storage Driver: windowsfilter
  Windows:
 Logging Driver: json-file
 Plugins:
  Volume: local
  Network: ics internal l2bridge l2tunnel nat null overlay private transparent
  Log: awslogs etwlogs fluentd gcplogs gelf json-file local logentries splunk syslog
 Swarm: inactive
 Default Isolation: hyperv
 Kernel Version: 10.0 19042 (19041.1.amd64fre.vb_release.191206-1406)
 Operating System: Windows 10 Pro Version 2004 (OS Build 19042.1288)
 OSType: windows
 Architecture: x86_64
 CPUs: 8
 Total Memory: 31.86GiB
 Name: plachta-REDACTED
 ID: PPOW:WCJM:ONM6:PQAX:TYOB:TILI:QVYT:DC4L:L3QD:4BFC:XQ3T:VLHB
 Docker Root Dir: C:\ProgramData\Docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  192.168.1.103:5000
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

Machine 2

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Build with BuildKit (Docker Inc., v0.6.1-docker)
  compose: Docker Compose (Docker Inc., v2.0.0-rc.3)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 887
 Server Version: 20.10.8
 Storage Driver: windowsfilter
  Windows:
 Logging Driver: json-file
 Plugins:
  Volume: local
  Network: ics internal l2bridge l2tunnel nat null overlay private transparent
  Log: awslogs etwlogs fluentd gcplogs gelf json-file local logentries splunk syslog
 Swarm: inactive
 Default Isolation: hyperv
 Kernel Version: 10.0 19043 (19041.1.amd64fre.vb_release.191206-1406)
 Operating System: Windows 10 Pro Version 2009 (OS Build 19043.1288)
 OSType: windows
 Architecture: x86_64
 CPUs: 10
 Total Memory: 31.25GiB
 Name: DESKTOP-4QJM6I4
 ID: T6YK:FX3X:TNIQ:QEZ2:H2CQ:5XDX:7WDO:ZZJB:UWSX:DAQP:SPGA:V2HS
 Docker Root Dir: C:\ProgramData\Docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  192.168.1.103:5000
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

Used GitLab Runner version

First tried with:

Version:      13.11.0
Git revision: 7f7a4bb0
Git branch:   13-11-stable
GO version:   go1.13.8
OS/Arch:      windows/amd64

After update tested with:

Version:      14.3.2
Git revision: e0218c92
Git branch:   14-3-stable
GO version:   go1.13.8
Built:        2021-09-30T16:11:30+0000
OS/Arch:      windows/amd64

Additional notes

Other gitlab runner helper images were tested. I tried combining some other images and jobs but issue seems to be caused by git in helper image.

Helper images tested:

x86_64-7f7a4bb0-servercore2004 (6 months ago)
x86_64-e0218c92-servercore2004 (18 days ago)
x86_64-c8933b98-servercore20H2 (6 days ago)
x86_64-e0218c92-servercore20H2 (18 days ago)

Content of directory mentioned by error message in cloned repo:

ls .\.git\refs\remotes\origin\


    Directory: C:\Users\plachta\connector\.git\refs\remotes\origin


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a----        10/18/2021  12:08 AM             30 HEAD

Edited Oct 19, 2021 by jan holcak