Gitlab Runner on Windows fails to check out repository from time to time
Summary
From time to time some of our jobs run into a state where they are not longer able to check out the repository. They are stuck before the job can even start.
This also happened with commits that previously run without problems (started again after two days and failed).
Sometimes the job is successful after clicking on Retry
but once the pipeline is in this "state" it's not possible to run the job from the pipeline trigger directly.
Forking the repository sometimes solves the problem.
Getting source from Git repository
00:08
Fetching changes with git depth set to 50...
fatal: bad config line 1 in file c:\builds\development\full_build\systemBuild.tmp\git-template\config
Cleaning up project directory and file based variables
00:08
ERROR: Job failed: exit code 128
Steps to reproduce
Problem occurs randomly, but only on windows containers.
.gitlab-ci.yml
build_windows:
needs: [fetch_app]
stage: build
image: docker.artifactory.customdomain.net/win_build_2019:latest
tags:
- docker
- windows_2019
before_script:
- echo "skip before"
script:
- echo "test"
Actual behavior
- Fails at getting source from Git repository
Expected behavior
- Starts job
Relevant logs and/or screenshots
job log
Running with gitlab-runner 14.4.0 (4b9e985a)
on winRunner2_full_build_docker 17_M7UM6
Resolving secrets
00:00
Preparing the "docker-windows" executor
00:01
Using Docker executor with image docker.artifactory.customdomain.net/win_build_2019:latest ...
Pulling docker image docker.artifactory.customdomain.net/win_build_2019:latest ...
Using docker image sha256:587cc0571e829030d9b61f7f82f85a880290bcdc7121bb2dc4747e8f9904fcba for docker.artifactory.customdomain.net/win_build_2019:latest with digest docker.artifactory.customdomain.net/win_build_2019:latest@sha256:8395e84bfb75259560bad5616f94d341c3d4f48bd1dd96964d4e87438e804d7b ...
Preparing environment
00:57
Running on RUNNER-17M7UM6- via
winRunner2...
Getting source from Git repository
01:13
Fetching changes with git depth set to 50...
fatal: bad config line 1 in file c:\builds\development\full_build\systemBuild.tmp\git-template\config
Cleaning up project directory and file based variables
00:49
ERROR: Job failed: exit code 128
Environment description
Custom installation of shared runner on windows 2019
config.toml contents
concurrent = 4
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "winRunner2_full_build_docker"
url = "https://gitlab.customdomain.net/"
token = "vAliDtOkEn-rEpLaceD"
tls-ca-file = "C:\\Dev\\ca.crt"
executor = "docker-windows"
environment = ["https_proxy=http://customdomain.net:10101/"]
shell = "powershell"
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
tls_verify = false
image = "PLACEHOLDER"
privileged = false
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["c:\\cache"]
shm_size = 0
Used GitLab Runner version
.\gitlab-runner.exe --version
Version: 14.4.0
Git revision: 4b9e985a
Git branch: 14-4-stable
GO version: go1.13.8
Built: 2021-10-25T14:42:45+0000
OS/Arch: windows/amd64
docker -v
Docker version 19.03.12, build 4306744