Pushing docker image to gitlab registry fails after 8.10 update
This breaks my company's workflow!
Summary
Pushing docker image during CI fails after 8.10 update. It worked fine before.
Steps to reproduce
In my .gitlab-ci.yml
file, I have:
- docker info
- docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $REGISTRY
- docker push $MY_REGISTRY/my-team/my-project
When the build runs, I get:
$ docker info
Containers: 97
Running: 0
Paused: 0
Stopped: 97
Images: 547
Server Version: 1.11.2
Storage Driver: devicemapper
Pool Name: docker-253:0-17847758-pool
Pool Blocksize: 65.54 kB
Base Device Size: 107.4 GB
Backing Filesystem: xfs
Data file: /dev/loop0
Metadata file: /dev/loop1
Data Space Used: 6.548 GB
Data Space Total: 107.4 GB
Data Space Available: 85.81 GB
Metadata Space Used: 36.1 MB
Metadata Space Total: 2.147 GB
Metadata Space Available: 2.111 GB
Udev Sync Supported: true
Deferred Removal Enabled: false
Deferred Deletion Enabled: false
Deferred Deleted Device Count: 0
Data loop file: /var/lib/docker/devicemapper/devicemapper/data
WARNING: Usage of loopback devices is strongly discouraged for production use. Either use `--storage-opt dm.thinpooldev` or use `--storage-opt dm.no_warn_on_loop_devices=true` to suppress this warning.
Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
Library Version: 1.02.107-RHEL7 (2016-06-09)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge null host
Kernel Version: 3.10.0-229.14.1.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 3.703 GiB
Name: git.company-name.com
ID: ZIDB:VMNS:2LTS:JH7G:TPB4:E5ZU:KIJ4:OCKI:AT5B:35NS:5ARY:QRMY
Docker Root Dir: /var/lib/docker
Debug mode (client): false
Debug mode (server): false
Registry: https://index.docker.io/v1/
$ docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $REGISTRY
Login Succeeded
$ docker push $MY_REGISTRY/my-team/my-project
The push refers to a repository [git.company-name.com:4567/my-team/my-project]
9bd2df9d9ac2: Preparing
d39154d08267: Preparing
14bf145d0917: Preparing
82c27186c092: Preparing
5f70bf18a086: Preparing
5f70bf18a086: Preparing
5f70bf18a086: Preparing
5f70bf18a086: Preparing
5f70bf18a086: Preparing
5f70bf18a086: Preparing
5f70bf18a086: Preparing
5f70bf18a086: Preparing
c2bff8368aa9: Preparing
335a9bc21e98: Preparing
5f70bf18a086: Preparing
5f70bf18a086: Preparing
8b7f6df7615d: Preparing
5f70bf18a086: Preparing
947564b0c8d9: Preparing
5f70bf18a086: Preparing
5f70bf18a086: Preparing
335a9bc21e98: Waiting
8b7f6df7615d: Waiting
947564b0c8d9: Waiting
c2bff8368aa9: Waiting
82c27186c092: Layer already exists
5f70bf18a086: Layer already exists
c2bff8368aa9: Layer already exists
335a9bc21e98: Layer already exists
14bf145d0917: Layer already exists
9bd2df9d9ac2: Retrying in 5 seconds
8b7f6df7615d: Layer already exists
947564b0c8d9: Layer already exists
d39154d08267: Retrying in 5 seconds
9bd2df9d9ac2: Retrying in 4 seconds
d39154d08267: Retrying in 4 seconds
9bd2df9d9ac2: Retrying in 3 seconds
d39154d08267: Retrying in 3 seconds
9bd2df9d9ac2: Retrying in 2 seconds
d39154d08267: Retrying in 2 seconds
9bd2df9d9ac2: Retrying in 1 second
d39154d08267: Retrying in 1 second
9bd2df9d9ac2: Retrying in 10 seconds
d39154d08267: Retrying in 10 seconds
9bd2df9d9ac2: Retrying in 9 seconds
d39154d08267: Retrying in 9 seconds
9bd2df9d9ac2: Retrying in 8 seconds
d39154d08267: Retrying in 8 seconds
9bd2df9d9ac2: Retrying in 7 seconds
d39154d08267: Retrying in 7 seconds
9bd2df9d9ac2: Retrying in 6 seconds
d39154d08267: Retrying in 6 seconds
9bd2df9d9ac2: Retrying in 5 seconds
d39154d08267: Retrying in 5 seconds
9bd2df9d9ac2: Retrying in 4 seconds
d39154d08267: Retrying in 4 seconds
9bd2df9d9ac2: Retrying in 3 seconds
d39154d08267: Retrying in 3 seconds
9bd2df9d9ac2: Retrying in 2 seconds
d39154d08267: Retrying in 2 seconds
9bd2df9d9ac2: Retrying in 1 second
d39154d08267: Retrying in 1 second
9bd2df9d9ac2: Retrying in 15 seconds
d39154d08267: Retrying in 15 seconds
9bd2df9d9ac2: Retrying in 14 seconds
d39154d08267: Retrying in 14 seconds
9bd2df9d9ac2: Retrying in 13 seconds
d39154d08267: Retrying in 13 seconds
9bd2df9d9ac2: Retrying in 12 seconds
d39154d08267: Retrying in 12 seconds
9bd2df9d9ac2: Retrying in 11 seconds
d39154d08267: Retrying in 11 seconds
9bd2df9d9ac2: Retrying in 10 seconds
d39154d08267: Retrying in 10 seconds
9bd2df9d9ac2: Retrying in 9 seconds
d39154d08267: Retrying in 9 seconds
9bd2df9d9ac2: Retrying in 8 seconds
d39154d08267: Retrying in 8 seconds
9bd2df9d9ac2: Retrying in 7 seconds
d39154d08267: Retrying in 7 seconds
9bd2df9d9ac2: Retrying in 6 seconds
d39154d08267: Retrying in 6 seconds
9bd2df9d9ac2: Retrying in 5 seconds
d39154d08267: Retrying in 5 seconds
9bd2df9d9ac2: Retrying in 4 seconds
d39154d08267: Retrying in 4 seconds
9bd2df9d9ac2: Retrying in 3 seconds
d39154d08267: Retrying in 3 seconds
9bd2df9d9ac2: Retrying in 2 seconds
d39154d08267: Retrying in 2 seconds
9bd2df9d9ac2: Retrying in 1 second
d39154d08267: Retrying in 1 second
9bd2df9d9ac2: Retrying in 20 seconds
d39154d08267: Retrying in 20 seconds
9bd2df9d9ac2: Retrying in 19 seconds
d39154d08267: Retrying in 19 seconds
9bd2df9d9ac2: Retrying in 18 seconds
d39154d08267: Retrying in 18 seconds
9bd2df9d9ac2: Retrying in 17 seconds
d39154d08267: Retrying in 17 seconds
9bd2df9d9ac2: Retrying in 16 seconds
d39154d08267: Retrying in 16 seconds
9bd2df9d9ac2: Retrying in 15 seconds
d39154d08267: Retrying in 15 seconds
9bd2df9d9ac2: Retrying in 14 seconds
d39154d08267: Retrying in 14 seconds
9bd2df9d9ac2: Retrying in 13 seconds
d39154d08267: Retrying in 13 seconds
9bd2df9d9ac2: Retrying in 12 seconds
d39154d08267: Retrying in 12 seconds
9bd2df9d9ac2: Retrying in 11 seconds
d39154d08267: Retrying in 11 seconds
9bd2df9d9ac2: Retrying in 10 seconds
d39154d08267: Retrying in 10 seconds
9bd2df9d9ac2: Retrying in 9 seconds
d39154d08267: Retrying in 9 seconds
9bd2df9d9ac2: Retrying in 8 seconds
d39154d08267: Retrying in 8 seconds
9bd2df9d9ac2: Retrying in 7 seconds
d39154d08267: Retrying in 7 seconds
9bd2df9d9ac2: Retrying in 6 seconds
d39154d08267: Retrying in 6 seconds
9bd2df9d9ac2: Retrying in 5 seconds
d39154d08267: Retrying in 5 seconds
9bd2df9d9ac2: Retrying in 4 seconds
d39154d08267: Retrying in 4 seconds
9bd2df9d9ac2: Retrying in 3 seconds
d39154d08267: Retrying in 3 seconds
9bd2df9d9ac2: Retrying in 2 seconds
d39154d08267: Retrying in 2 seconds
9bd2df9d9ac2: Retrying in 1 second
d39154d08267: Retrying in 1 second
blob upload unknown
ERROR: Build failed: exit status 1
Expected behavior
No errors when pushing image, which was behaviour previous to 8.10
Output of checks
Results of GitLab Application Check
Checking GitLab Shell ...
GitLab Shell version >= 3.2.0 ? ... OK (3.2.0)
Repo base directory exists?
default... yes
Repo storage directories are symlinks?
default... no
Repo paths owned by git:git?
default... yes
Repo paths access is drwxrws---?
default... yes
hooks directories in repos are links: ...
(list of projects with ... ok)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Check GitLab API access: OK
Check directories and files:
/var/opt/gitlab/.ssh/authorized_keys: OK
Send ping to redis server: gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Sidekiq ...
Running? ... yes
Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking Reply by email ...
Reply by email is disabled in config/gitlab.yml
Checking Reply by email ... Finished
Checking LDAP ...
LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab ...
Git configured with autocrlf=input? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config outdated? ... no
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory setup correctly? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
projects have namespace: ...
(list of projects with ... yes)
Redis version >= 2.8.0? ... yes
Ruby version >= 2.1.0 ? ... yes (2.1.8)
Your git bin path is "/opt/gitlab/embedded/bin/git"
Git version >= 2.7.3 ? ... yes (2.7.4)
Active users: 12
Checking GitLab ... Finished
Results of GitLab Environment Info
System information
System: CentOS 7.2.1511
Current User: git
Using RVM: no
Ruby Version: 2.1.8p440
Gem Version: 2.5.1
Bundler Version:1.10.6
Rake Version: 10.5.0
Sidekiq Version:4.1.4
GitLab information
Version: 8.10.0
Revision: 9dec995
Directory: /opt/gitlab/embedded/service/gitlab-rails
DB Adapter: postgresql
URL: https://git.company-name.com
HTTP Clone URL: https://git.company-name.com/some-group/some-project.git
SSH Clone URL: git@git.company-name.com:some-group/some-project.git
Using LDAP: no
Using Omniauth: no
GitLab Shell
Version: 3.2.0
Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories
Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks/
Git: /opt/gitlab/embedded/bin/git
Possible fixes
None known