fatal: destination path '/my/path' already exists and is not an empty directory.
Summary
I have two jobs. First job displays information about the repository. The second job clones another repository because it's needed as a dependency to compile the code. The first time a pipeline is created, it completes both jobs successfully. The second time a pipeline is created, the second job fails because the directory where the dependency repository is cloned into is already there, from the first pipeline. Thereby generating the error: fatal: destination path '/my/path' already exists and is not an empty directory.
Steps to reproduce
Create a job that clones a repository into the /builds directory. Run it twice, and the second time should fail since the cloned repo is still present.
.gitlab-ci.yml
stage: dependencies
script:
- git clone https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.com/staging-fw/my_dependencies.git /builds/dependencies
Actual behavior
The second time a pipeline is created, the /builds directory already contains the /builds/dependencies folder.
Expected behavior
Since I'm using docker, my expectation is that the Docker container created for the pipeline will be blank and not contain any previous folders/files that were used in previous containers. That is obviously not the case since the /builds directory used by gitlab-runner contains previous content and is shared among every pipeline that is created.
Relevant logs and/or screenshots
job log
on old_gen_runner 6fsjs96e
Using Docker executor with image gcc-arm_4_7-2013q3:3.0 ...
Using docker image sha256:5d4741a428654beb44a3c004822e4d2ceededc88f22ec1ec7f45dedf707a0302 for gcc-arm_4_7-2013q3:3.0 ...
Running on runner-6fsjs96e-project-13664495-concurrent-0 via my_laptop.local...
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/staging-fw/my-project/.git/
Created fresh repository.
From https://gitlab.com/staging-fw/my-project
* [new branch] master -> origin/master
Checking out 139f4bdd as master...
Skipping Git submodules setup
$ python /scripts/info.py
Printing job info...
Builds dir: /builds
Commit Message:
Modified yml file.
Branch: master
Project dir: /builds/staging-fw/my-project
$ ls /builds
my-project
$ git clone https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.com/staging-fw/my_dependencies.git /builds/dependencies
Cloning into '/builds/dependencies'...
Job succeeded
One thing to note is that the first time a pipeline is created it displays:
Initialized empty Git repository
The second time a pipeline is created, the log output is as follows:
Running with gitlab-runner 12.2.0 (a987417a)
on old_gen_runner 6fsjs96e
Using Docker executor with image gcc-arm_4_7-2013q3:3.0 ...
Using docker image sha256:5d4741a428654beb44a3c004822e4d2ceededc88f22ec1ec7f45dedf707a0302 for gcc-arm_4_7-2013q3:3.0 ...
Running on runner-6fsjs96e-project-13664495-concurrent-0 via my-laptop.local...
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /builds/staging-fw/my-project/.git/
From https://gitlab.com/staging-fw/my-project
722c2c3..f321b75 master -> origin/master
Checking out f321b75f as master...
Skipping Git submodules setup
$ python /scripts/info.py
Printing job info...
Builds dir: /builds
Commit Message:
Modified yml file.
Branch: master
Project dir: /builds/staging-fw/my-project
$ ls /builds
my-project
tools
This time the log output said:
Reinitialized existing Git repository
And this time it listed two folders in the /builds directory:
$ ls /builds
staging-fw
tools
This indicates to me that the /builds directory is not created individually for each Docker container and is in fact shared between all pipelines. Is this correct? Because I can't find any documentation on this.
Environment description
I am using gitlab-runner and Docker on my MacBook Pro running Mojave v10.14.6.
My repos are hosted on GitLab.com.
gitlab-runner:
Version: 12.2.0 Git revision: a987417a Git branch: 12-2-stable GO version: go1.8.7 Built: 2019-08-22T13:06:00+0000 OS/Arch: darwin/amd64
docker:
Docker version 19.03.1, build 74b1e89
Client: Debug Mode: false
Server: Containers: 5 Running: 0 Paused: 0 Stopped: 5 Images: 36 Server Version: 19.03.1 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true Logging Driver: json-file Cgroup Driver: cgroupfs Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: runc Default Runtime: runc Init Binary: docker-init containerd version: 894b81a4b802e4eb2a91d1ce216b8817763c29fb runc version: 425e105d5a03fabd737a126ad93d62a9eeede87f init version: fec3683 Security Options: seccomp Profile: default Kernel Version: 4.9.184-linuxkit Operating System: Docker Desktop OSType: linux Architecture: x86_64 CPUs: 4 Total Memory: 1.952GiB Name: docker-desktop ID: DGZ6:2TO4:OFDJ:MTXA:DUWZ:R3ZN:KWGA:F5UJ:Z4RM:2ABB:53O5:F3RL Docker Root Dir: /var/lib/docker Debug Mode: true File Descriptors: 30 Goroutines: 46 System Time: 2019-09-04T00:12:39.5088795Z EventsListeners: 2 HTTP Proxy: gateway.docker.internal:3128 HTTPS Proxy: gateway.docker.internal:3129 Registry: https://index.docker.io/v1/ Labels: Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false Product License: Community Engine
config.toml contents
concurrent = 1
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "old_gen_runner"
url = "https://gitlab.com"
token = "6xxxxxxxxxxxxxxxxx"
executor = "docker"
[runners.custom_build_dir]
[runners.docker]
tls_verify = false
image = "gcc:5.2.0"
privileged = false
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
pull_policy = "never"
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
Used GitLab Runner version
gitlab-runner:
Version: 12.2.0
Git revision: a987417a
Git branch: 12-2-stable
GO version: go1.8.7
Built: 2019-08-22T13:06:00+0000
OS/Arch: darwin/amd64
Using Docker version 19.03.1, build 74b1e89 executor with custom image gcc-arm_4_7-2013q3:3.0
-->
Possible fixes
I don't have a recommendation for a fix, I'm simply trying to understand why this is happening. If this is expected then it's fine, I just want to understand why and how. Thank you.