fix(datastore): race condition pushing two or more manifests by tag with the same digest
Summary
I am getting an error when trying to build a docker image and publish it to container registry
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':jib'.
> com.google.cloud.tools.jib.plugins.common.BuildStepsExecutionException: 500 Internal Server Error
{"errors":[{"code":"UNKNOWN","message":"unknown error","detail":{}}]}
We have more than 50 projects in the repository and most of them are working fine. They are all built the same way. This issue affects new projects.
Steps to reproduce
- create new project in gitlab (ex.
test-project-1
) - get working project (which has successful pipeline) and add origin from
test-project-1
- push project
- pipeline fails
Example Project
What is the current bug behavior?
Pipeline failed.
I noticed some interesting logs after enabling debug mode in jib
.
As I mentioned before - there is a problem only with new projects (with the same dependencies like jib, spring, java version etc.).
Relevant logs and/or screenshots
2022-03-18T16:58:44.764+0000 [INFO] [com.google.api.client.http.HttpTransport] -------------- REQUEST --------------
PUT https://registry.gitlab.com/v2/<group>/<project>/manifests/clean-project
Accept:
Accept-Encoding: gzip
Authorization: <Not Logged>
User-Agent: jib 3.2.0 jib-gradle-plugin Google-HTTP-Java-Client/1.34.0 (gzip)
Content-Type: application/vnd.docker.distribution.manifest.v2+json
2022-03-18T16:58:44.764+0000 [INFO] [com.google.api.client.http.HttpTransport] curl -v --compressed -X PUT -H 'Accept: ' -H 'Accept-Encoding: gzip' -H 'Authorization: <Not Logged>' -H 'User-Agent: jib 3.2.0 jib-gradle-plugin Google-HTTP-Java-Client/1.34.0 (gzip)' -H 'Content-Type: application/vnd.docker.distribution.manifest.v2+json' -d '@-' -- 'https://registry.gitlab.com/v2/<group>/<project>/manifests/clean-project' << $$$
2022-03-18T16:58:44.797+0000 [DEBUG] [jdk.event.security] ValidationChain: -347365895, -1835097136, -1473757959
2022-03-18T16:58:44.799+0000 [DEBUG] [jdk.event.security] ValidationChain: -347365895, -1835097136, -1473757959
2022-03-18T16:58:44.802+0000 [DEBUG] [jdk.event.security] TLSHandshake: registry.gitlab.com:443, TLSv1.3, TLS_AES_256_GCM_SHA384, -1473757959
2022-03-18T16:58:44.803+0000 [DEBUG] [jdk.event.security] TLSHandshake: registry.gitlab.com:443, TLSv1.3, TLS_AES_256_GCM_SHA384, -1473757959
2022-03-18T16:58:44.806+0000 [INFO] [com.google.api.client.http.HttpTransport] Total: 1,403 bytes
2022-03-18T16:58:44.806+0000 [INFO] [com.google.api.client.http.HttpTransport] {"schemaVersion":2,"mediaType":"application/vnd.docker.distribution.manifest.v2+json","config":{"mediaType":"application/vnd.docker.container.image.v1+json","digest":"sha256:c62a83f5a8b0cc102cefda11e3343e2709de5b4070e7612f42c3bdaeee8e35e1","size":6008},"layers":[{"mediaType":"application/vnd.docker.image.rootfs.diff.tar.gzip","digest":"sha256:36ccefbf3d8a9a1b18baaa9cbf0f3ad50e8a7b751656c74df359900a318cbffc","size":2816169},{"mediaType":"application/vnd.docker.image.rootfs.diff.tar.gzip","digest":"sha256:98cdb5547088b654845fc6a44678898e6d0af07d4301b4ceb54c62175b128947","size":6486650},{"mediaType":"application/vnd.docker.image.rootfs.diff.tar.gzip","digest":"sha256:f9543a44faf48a8db5867137bb2ecf9d154016c8281823a97526d8493aa603c7","size":5094},{"mediaType":"application/vnd.docker.image.rootfs.diff.tar.gzip","digest":"sha256:b73661c4b41c93232bb8f747bf51317c3a0d1abbc36c7bdf8496d210656e7864","size":138626713},{"mediaType":"application/vnd.docker.image.rootfs.diff.tar.gzip","digest":"sha256:bf5bec95ba76db36fafe8e3ebac630dc4cf7ab763a5442d104bc8a8756ab7360","size":7581779},{"mediaType":"application/vnd.docker.image.rootfs.diff.tar.gzip","digest":"sha256:e0240c0a3ceee5ce8fc9fad9518dd1b4794d0bdb79da2bd9dd8d741c625566e9","size":945},{"mediaType":"application/vnd.docker.image.rootfs.diff.tar.gzip","digest":"sha256:7e75240879d4c8b9816b89b59fa09cdbb0a59e2e0189a95a008d508fc20a6fda","size":401}]}
2022-03-18T16:58:44.807+0000 [INFO] [com.google.api.client.http.HttpTransport] Total: 1,403 bytes
2022-03-18T16:58:44.807+0000 [INFO] [com.google.api.client.http.HttpTransport] {"schemaVersion":2,"mediaType":"application/vnd.docker.distribution.manifest.v2+json","config":{"mediaType":"application/vnd.docker.container.image.v1+json","digest":"sha256:c62a83f5a8b0cc102cefda11e3343e2709de5b4070e7612f42c3bdaeee8e35e1","size":6008},"layers":[{"mediaType":"application/vnd.docker.image.rootfs.diff.tar.gzip","digest":"sha256:36ccefbf3d8a9a1b18baaa9cbf0f3ad50e8a7b751656c74df359900a318cbffc","size":2816169},{"mediaType":"application/vnd.docker.image.rootfs.diff.tar.gzip","digest":"sha256:98cdb5547088b654845fc6a44678898e6d0af07d4301b4ceb54c62175b128947","size":6486650},{"mediaType":"application/vnd.docker.image.rootfs.diff.tar.gzip","digest":"sha256:f9543a44faf48a8db5867137bb2ecf9d154016c8281823a97526d8493aa603c7","size":5094},{"mediaType":"application/vnd.docker.image.rootfs.diff.tar.gzip","digest":"sha256:b73661c4b41c93232bb8f747bf51317c3a0d1abbc36c7bdf8496d210656e7864","size":138626713},{"mediaType":"application/vnd.docker.image.rootfs.diff.tar.gzip","digest":"sha256:bf5bec95ba76db36fafe8e3ebac630dc4cf7ab763a5442d104bc8a8756ab7360","size":7581779},{"mediaType":"application/vnd.docker.image.rootfs.diff.tar.gzip","digest":"sha256:e0240c0a3ceee5ce8fc9fad9518dd1b4794d0bdb79da2bd9dd8d741c625566e9","size":945},{"mediaType":"application/vnd.docker.image.rootfs.diff.tar.gzip","digest":"sha256:7e75240879d4c8b9816b89b59fa09cdbb0a59e2e0189a95a008d508fc20a6fda","size":401}]}
2022-03-18T16:58:44.856+0000 [INFO] [com.google.api.client.http.HttpTransport] -------------- RESPONSE --------------
HTTP/1.1 500 Internal Server Error
Content-Type: application/json
Docker-Distribution-Api-Version: registry/2.0
Gitlab-Migration-Path: new
X-Content-Type-Options: nosniff
Date: Fri, 18 Mar 2022 16:58:44 GMT
Content-Length: 70
2022-03-18T16:58:44.856+0000 [INFO] [com.google.api.client.http.HttpTransport] Total: 70 bytes
2022-03-18T16:58:44.856+0000 [INFO] [com.google.api.client.http.HttpTransport] {"errors":[{"code":"UNKNOWN","message":"unknown error","detail":{}}]}
Status
Current as of 2022-04-08 #640 (comment 905432125)
Edited by Jaime Martinez