pushing windows native docker images to gitlab.com fails with "manifest blob unknown"
Summary
I built this windows native docker image, which I have no problem to push to dockerhub
https://hub.docker.com/r/shuliyey/iistest/
But when I'm pushing to gitlab it fails with the below error manifest blob unknown: blob unknown to registry
PS C:\Users\Caretaker\docker\radiancewebiis_base> .\scripts\gitlab\dockerpushinteractive.bat
Please specify the tag version of the radiancewebiis_base docker image to be pushed (gitlab): v0.0.1
Username (shuliyey):
Password:
Login Succeeded
The push refers to a repository [registry.gitlab.com/ethanwilansky/radiancewebiis_base]
c3c07c7bdd4e: Layer already exists
f388996841db: Layer already exists
2164f5d63ab6: Layer already exists
59d2aca59d06: Layer already exists
bb41028b9baa: Layer already exists
2163c8d45f61: Layer already exists
1392c0b78f60: Layer already exists
6c2b7edcf0e9: Layer already exists
e657f524e872: Layer already exists
81a36ef851f8: Layer already exists
022f684f98b3: Layer already exists
5fef0c0d74a9: Layer already exists
37e9891c49f7: Layer already exists
153f07ff407e: Layer already exists
7b62c67a952e: Layer already exists
983b098c4371: Layer already exists
f0d8bb058746: Layer already exists
3fd27ecef6a3: Skipped foreign layer
manifest blob unknown: blob unknown to registry
Steps to reproduce
- Make sure you have a Windows Server 2016 VM/Physical machine and set up windows native docker - https://msdn.microsoft.com/en-us/virtualization/windowscontainers/quick_start/quick_start_windows_server
- You need to open command line prompt in administrator mode
- The docker version I'm using is
PS C:\Users\Caretaker\docker\radiancewebiis_base> docker version
Client:
Version: 1.12.2-cs2-ws-beta-rc1
API version: 1.25
Go version: go1.7.1
Git commit: 62d9ff9
Built: Fri Sep 23 20:50:29 2016
OS/Arch: windows/amd64
Server:
Version: 1.12.2-cs2-ws-beta-rc1
API version: 1.25
Go version: go1.7.1
Git commit: 62d9ff9
Built: Fri Sep 23 20:50:29 2016
OS/Arch: windows/amd64
- Pull the shuliyey/iistest:vtest docker image, you have to use the windows native docker
docker pull shuliyey/iistest:vtest
- Tag it with a gitlab repository
docker tag shuliyey/iistest:vtest registry.gitlab.com/<user>/<repo>:<tag>
- Make sure you create the gitlab repo used to test this before pushing
- push the gitlab docker registry
docker login registry.gitlab.com
docker push registry.gitlab.com/<user>/<repo>:<tag>
Expected behavior
It should succeed, just like how it is pushed to dockerhub
PS C:\Users\Caretaker\docker\radiancewebiis_base> docker push shuliyey/iistest:vtest
The push refers to a repository [docker.io/shuliyey/iistest]
c3c07c7bdd4e: Pushed
...
f0d8bb058746: Mounted from microsoft/iis
3fd27ecef6a3: Skipped foreign layer
vtest: digest: sha256:e40ef8c7177fe79f5eadad6aad22d93faca1b794d69c4e8381fd50d1ec88dbfc size: 4253
or pushed to private docker registry we set up locally
PS C:\Users\Caretaker\docker\radiancewebiis_base> docker push registry.mev-rancher.dev.tech.local/radiance/radiancew
ebiis_base:v0.0.1
The push refers to a repository [registry.mev-rancher.dev.tech.local/radiance/radiancewebiis_base]
c3c07c7bdd4e: Layer already exists
...
f0d8bb058746: Layer already exists
3fd27ecef6a3: Skipped foreign layer
v0.0.1: digest: sha256:e40ef8c7177fe79f5eadad6aad22d93faca1b794d69c4e8381fd50d1ec88dbfc size: 4253
Actual behavior
You get the manifest blob unkown: blob unkown to registry behaviour
PS C:\Users\Caretaker\docker\radiancewebiis_base> .\scripts\gitlab\dockerpushinteractive.bat
Please specify the tag version of the radiancewebiis_base docker image to be pushed (gitlab): v0.0.1
Username (shuliyey):
Password:
Login Succeeded
The push refers to a repository [registry.gitlab.com/ethanwilansky/radiancewebiis_base]
c3c07c7bdd4e: Layer already exists
...
3fd27ecef6a3: Skipped foreign layer
manifest blob unknown: blob unknown to registry
Relevant logs and/or screenshots
Output of checks
I'm using the gitlab.com
Results of GitLab application Check
I'm using the gitlab.com
Results of GitLab environment info
I'm using the gitlab.com
Possible fixes
I believe docker-registry made a change to the way manifest is generated and handled from docker 1.10. I'm pasting some links
- http://stackoverflow.com/questions/35305320/manifest-invalid-error-while-pushing-an-image-in-bluemix
- https://github.com/docker/distribution/issues/928
I believe if the embedded docker registry in gitlab is upgraded to the latest version. This problem should get fix. But this is just a guess.
Because for us locally, we were using distribution/docker-registry:2.2.1, and we get the error - manifest invalid: manifest invalid, which is a difference error message (but similar to the one we're getting for gitlab.com)
After we upgraded to distribution/docker-registry:master (latest version). Everything worked. and we got the success message
v0.0.1: digest: sha256:e40ef8c7177fe79f5eadad6aad22d93faca1b794d69c4e8381fd50d1ec88dbfc size: 4253