pushing manifest to registry gets error "manifest blob unknown"
Summary
Pushing a docker multi-arch manifest list errors out with
manifest blob unknown: blob unknown to registry
Steps to reproduce
- Create several multi-arch images (
amd64
,arm64v8
,arm32v7
, etc)- Naming scheme
$IMAGE/$ARCH:$COMMIT
(ie,registry.gitlab.com/paleozogt/dockerfun/amd64:e99c4336
). - Dockerfile:
ARG ARCH FROM $ARCH/ubuntu:bionic
- Naming scheme
- Push the images
- Use
docker manifest create
to pull all images together into an image named$IMAGE:$COMMIT
(ie,registry.gitlab.com/paleozogt/dockerfun:e99c4336
) - Use
docker manifest push
to push the manifest list
Curiously, if we use a different naming scheme of $IMAGE:$ARCH-$COMMIT
(ie, registry.gitlab.com/paleozogt/dockerfun:amd64-6b972be0
) it will work.
Example Project
An example project showing the problem is here:
- Branch with multi-arch failing:
- Branch with multi-arch working:
What is the current bug behavior?
Pushing the multi-arch manifest fails even thought the images are in the repository.
What is the expected correct behavior?
Pushing the multi-arch manifest should succeed no matter the naming scheme used, as long as the images are in the repository.
Relevant logs and/or screenshots
$ docker manifest push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
Pushed ref registry.gitlab.com/paleozogt/dockerfun@sha256:05f7a02dff801b3c0e18b9588b63772f647aa068f61bca0b77d547525c01f571 with digest: sha256:1e43bde8fda64de203881785cbd7be4efe7cba4f6dbd435a21f9423762fae6f1
Pushed ref registry.gitlab.com/paleozogt/dockerfun@sha256:0b5c73ae2be5480753ccffc7c585c4526de2c8f01f81c2276bc96e7cfcd5c3b6 with digest: sha256:34962a4a4fc7a3134cd671a4c137eb4945f4361cab4e24341685828ee5187964
Pushed ref registry.gitlab.com/paleozogt/dockerfun@sha256:0da3d97dc96540fd02918d63157911168cd660119c02f91150621afdf85e1668 with digest: sha256:00fab59b2048a51ef7c49785effffbcfd9eff7aadaa55e08cf03f1b0cd60431a
failed to put manifest registry.gitlab.com/paleozogt/dockerfun:e99c4336: errors:
manifest blob unknown: blob unknown to registry
manifest blob unknown: blob unknown to registry
manifest blob unknown: blob unknown to registry
ERROR: Job failed: exit code 1
Output of checks
This bug happens on GitLab.com as well as my Omnibus installation.