gitlab-runner 13.2.1 (efa30e33) docker executor fails to lock /root/.gitconfig
Summary
That is weird thing on one host which seems happens inside runner's docker container
What is the current bug behavior?
Running with gitlab-runner 13.2.1 (efa30e33)
on klirichek-desktop 4j6WjyEZ
Preparing the "docker" executor
00:04
Using Docker executor with image registry.gitlab.com/manticoresearch/dev/ubertests_public:latest ...
Authenticating with credentials from job payload (GitLab Registry)
Pulling docker image registry.gitlab.com/manticoresearch/dev/ubertests_public:latest ...
Using docker image sha256:1ea753b2eb280c12b36c97dba7d6d84ccf1040fc637b84c6e949639c3c677698 for registry.gitlab.com/manticoresearch/dev/ubertests_public:latest ...
Preparing environment
00:02
Running on runner-4j6wjyez-project-3858465-concurrent-0 via ubuntu64...
Getting source from Git repository
00:02
error: could not lock config file /root/.gitconfig: Operation not permitted
Uploading artifacts for failed job
00:04
Uploading artifacts...
build/xml_release_tests: found 5 matching files and directories
build/test/test_*/report.txt: found 341 matching files and directories
build/test/error*.txt: found 11 matching files and directories
build/test/searchd.log: found 1 matching files and directories
build/test/examples.txt: found 1 matching files and directories
Uploading artifacts as "archive" to coordinator... ok id=664286121 responseStatus=201 Created token=6zCjeQsV
ERROR: Job failed: exit code 1
Weird lines are:
Getting source from Git repository
error: could not lock config file /root/.gitconfig: Operation not permitted
The pipeline job is failed because of it.
What is the expected correct behavior?
Running with gitlab-runner 13.1.1 (6fbc7474)
on dev-docker-2 48J_msE_
Preparing the "docker" executor
00:16
Using Docker executor with image registry.gitlab.com/manticoresearch/dev/ubertests_public:latest ...
Using locally found image version due to if-not-present pull policy
Using docker image sha256:1ea753b2eb280c12b36c97dba7d6d84ccf1040fc637b84c6e949639c3c677698 for registry.gitlab.com/manticoresearch/dev/ubertests_public:latest ...
Preparing environment
00:10
Running on runner-48jmse-project-3858465-concurrent-0 via dev.manticoresearch.com...
Getting source from Git repository
00:17
Fetching changes...
Reinitialized existing Git repository in /builds/manticoresearch/dev/.git/
Checking out 8a3351e9 as master...
Removing build/
Removing cache/
Removing manticore-3.5.1_200730.8a3351e-1.el6.x86_64.rpm
Removing manticore-converter-3.5.1_200730.8a3351e-1.el6.x86_64.rpm
Removing manticore-debuginfo-3.5.1_200730.8a3351e-1.el6.x86_64.rpm
... (etc.)
Other details
Runner's section in config (with sensitive data wiped away)
[[runners]]
name = "klirichek-desktop"
limit = 1
url = "https://gitlab.com"
token = "...."
executor = "docker"
[runners.docker]
tls_verify = false
image = "alpine:latest"
privileged = false
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
pull_policy = "if-not-present"
shm_size = 0
[runners.cache]
Type = "s3"
Path = "gitlab"
Shared = true
[runners.cache.s3]
ServerAddress = .....
AccessKey = .....
SecretKey = .....
BucketName = "runner-cache"
Insecure = true
[runners.cache.gcs]
The successful host has absolutely same config, except host name (which is dev-docker-2 there).
I've tried investigate it localy, starting gitlab-runner --debug
, but it gives no extra result around this failing line.
- image registry.gitlab.com/manticoresearch/dev/ubertests_public is corehent on both hosts.
- there is no /root/.gitconfig in that image, so it looks like shit happens in gitlab-runner volume
- note - failing is 13.2.1 (efa30e33). Successing is 13.1.1 (6fbc7474). Problem appears exactly after
apt upgrade
on the host, which upgraded gitlab-runner.
I've tried:
- stop and wipe out all dockers (ensure that
docker ps -a
shows nothing) - wipe out with
docker rmi
all images with linegitlab
,runner
,alpine
inside.
Nothing helped.
It seems, that the image is not very 'ephemeral', as I see in success run lines like:
Reinitialized existing Git repository in /builds/manticoresearch/dev/.git/
Checking out 8a3351e9 as master...
I just wonder: if it runs docker executor 'from scratch' - where this 'existing Git repository' resides? As config of the runner provides no shared volumes with the host. May be wiping out this traces will help, but I just don't know where to find them...