ERROR: Preparation failed: Error response from daemon: Conflict. The container name ... You have to remove (or rename) that container to be able to reuse that name. (executor_docker.go:709:0s)

Summary

Random jobs fail with error:

ERROR: Preparation failed: Error response from daemon: Conflict. The container name "/runner-Jqo2y6XZ-project-23-concurrent-0-docker-0" is already in use by container "26b20f2f80f52b47c3ae6595a6b83e0c9c94456ca04700a04fa33761328d40c6". You have to remove (or rename) that container to be able to reuse that name. (executor_docker.go:709:0s)

Steps to reproduce

Happens randomly.

Example Project

None, since we run on premise Gitlab EE version.

What is the current bug behavior?

When trying to run a job, we encounter this error.

What is the expected correct behavior?

The job should run.

Relevant logs and/or screenshots

Full error:

Running with gitlab-runner 11.8.0 (4745a6f3) on i-09e8604903824a119 Jqo2y6XZ Using Docker executor with image x/cicd-base ... Starting service docker:dind ... Pulling docker image docker:dind ... Using docker image sha256:85e924caedbd3e5245ad95cc7471168e923391b22dcb559decebe4a378a06939 for docker:dind ... ERROR: Preparation failed: Error response from daemon: Conflict. The container name "/runner-Jqo2y6XZ-project-23-concurrent-0-docker-0" is already in use by container "26b20f2f80f52b47c3ae6595a6b83e0c9c94456ca04700a04fa33761328d40c6". You have to remove (or rename) that container to be able to reuse that name. (executor_docker.go:709:0s) Will be retried in 3s ... Using Docker executor with image x/cicd-base ... Starting service docker:dind ... Pulling docker image docker:dind ... Using docker image sha256:85e924caedbd3e5245ad95cc7471168e923391b22dcb559decebe4a378a06939 for docker:dind ... ERROR: Preparation failed: Error response from daemon: Conflict. The container name "/runner-Jqo2y6XZ-project-23-concurrent-0-docker-0" is already in use by container "26b20f2f80f52b47c3ae6595a6b83e0c9c94456ca04700a04fa33761328d40c6". You have to remove (or rename) that container to be able to reuse that name. (executor_docker.go:709:0s) Will be retried in 3s ... Using Docker executor with image x/cicd-base ... Starting service docker:dind ... Pulling docker image docker:dind ... Using docker image sha256:85e924caedbd3e5245ad95cc7471168e923391b22dcb559decebe4a378a06939 for docker:dind ... ERROR: Preparation failed: Error response from daemon: Conflict. The container name "/runner-Jqo2y6XZ-project-23-concurrent-0-docker-0" is already in use by container "26b20f2f80f52b47c3ae6595a6b83e0c9c94456ca04700a04fa33761328d40c6". You have to remove (or rename) that container to be able to reuse that name. (executor_docker.go:709:0s) Will be retried in 3s ... ERROR: Job failed (system failure): Error response from daemon: Conflict. The container name "/runner-Jqo2y6XZ-project-23-concurrent-0-docker-0" is already in use by container "26b20f2f80f52b47c3ae6595a6b83e0c9c94456ca04700a04fa33761328d40c6". You have to remove (or rename) that container to be able to reuse that name. (executor_docker.go:709:0s)

Output of checks

None

Results of GitLab environment info

System information
System:		Ubuntu 16.04
Proxy:		no
Current User:	git
Using RVM:	no
Ruby Version:	2.5.3p105
Gem Version:	2.7.6
Bundler Version:1.16.6
Rake Version:	12.3.2
Redis Version:	3.2.12
Git Version:	2.18.1
Sidekiq Version:5.2.5
Go Version:	unknown
GitLab information
Version:	11.8.1-ee
Revision:	39d0b2e
Directory:	/opt/gitlab/embedded/service/gitlab-rails
DB Adapter:	postgresql
DB Version:	9.6.11
URL:		x
HTTP Clone URL:	x
SSH Clone URL:	x
Elasticsearch:	yes
Geo:		no
Using LDAP:	no
Using Omniauth:	yes
Omniauth Providers:
GitLab Shell
Version:	8.4.4
Repository storage paths:

• default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git

Results of GitLab application Check

Checking GitLab subtasks ...
Checking GitLab Shell ...
GitLab Shell: ... GitLab Shell version >= 8.4.4 ? ... OK (8.4.4)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Check GitLab API access: OK
Redis available via internal API: OK
Access to /var/opt/gitlab/.ssh/authorized_keys: OK
gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Gitaly ...
Gitaly: ... default ... OK
Checking Gitaly ... Finished
Checking Sidekiq ...
Sidekiq: ... Running? ... yes
Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking Incoming Email ...
Incoming Email: ... Reply by email is disabled in config/gitlab.yml
Checking Incoming Email ... Finished
Checking LDAP ...
LDAP: ... LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab App ...
Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
16/3 ... yes
15/5 ... yes
5/8 ... yes
59/9 ... yes
59/10 ... yes
5/12 ... yes
20/13 ... yes
61/15 ... yes
61/16 ... yes
5/17 ... yes
3/18 ... yes
4/21 ... yes
5/23 ... yes
4/24 ... yes
5/26 ... yes
75/27 ... yes
76/28 ... yes
77/29 ... yes
75/30 ... yes
75/31 ... yes
75/32 ... yes
75/33 ... yes
75/35 ... yes
5/36 ... yes
5/37 ... yes
5/38 ... yes
5/39 ... yes
5/40 ... yes
5/41 ... yes
5/42 ... yes
5/43 ... yes
5/44 ... yes
5/45 ... yes
5/46 ... yes
5/47 ... yes
5/48 ... yes
5/49 ... yes
5/50 ... yes
3/53 ... yes
5/54 ... yes
5/55 ... yes
5/56 ... yes
80/57 ... yes
5/58 ... yes
5/59 ... yes
5/60 ... yes
5/61 ... yes
5/62 ... yes
59/63 ... yes
5/65 ... yes
5/66 ... yes
5/67 ... yes
5/68 ... yes
4/69 ... yes
81/70 ... yes
5/72 ... yes
3/73 ... yes
5/74 ... yes
5/75 ... yes
5/80 ... yes
3/81 ... yes
3/82 ... yes
3/83 ... yes
3/84 ... yes
5/85 ... yes
5/86 ... yes
84/89 ... yes
84/90 ... yes
84/91 ... yes
84/92 ... yes
84/93 ... yes
84/95 ... yes
84/96 ... yes
84/97 ... yes
84/98 ... yes
58/100 ... yes
15/102 ... yes
58/103 ... yes
90/105 ... yes
90/106 ... yes
90/107 ... yes
90/109 ... yes
20/110 ... yes
90/111 ... yes
90/112 ... yes
90/113 ... yes
90/114 ... yes
90/115 ... yes
17/116 ... yes
16/117 ... yes
84/118 ... yes
59/119 ... yes
4/120 ... yes
84/121 ... yes
5/122 ... yes
5/123 ... yes
5/125 ... yes
5/126 ... yes
5/127 ... yes
104/129 ... yes
104/130 ... yes
104/131 ... yes
104/132 ... yes
85/133 ... yes
85/134 ... yes
6/136 ... yes
5/169 ... yes
150/170 ... yes
5/172 ... yes
5/173 ... yes
85/174 ... yes
15/175 ... yes
5/176 ... yes
150/177 ... yes
82/178 ... yes
17/179 ... yes
4/180 ... yes
5/181 ... yes
4/182 ... yes
5/185 ... yes
16/186 ... yes
15/188 ... yes
17/189 ... yes
5/190 ... yes
104/191 ... yes
5/192 ... yes
5/193 ... yes
15/195 ... yes
15/196 ... yes
59/197 ... yes
16/198 ... yes
105/200 ... yes
84/201 ... yes
5/202 ... yes
5/203 ... yes
5/204 ... yes
16/205 ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.3.5 ? ... yes (2.5.3)
Git version >= 2.18.0 ? ... yes (2.18.1)
Git user has default SSH configuration? ... yes
Active users: ... 87
Elasticsearch version 5.6 - 6.x? ... yes (6.3.1)
Checking GitLab App ... Finished
Checking GitLab subtasks ... Finished

Possible fixes

Seems to happen mostly on longer running runners.