gitlab docker registry cannot read or write to the minio s3
Summary
In this setup, concerning the registry and minio, everything has been deployed with the default settings and over the helm chart. Everything else is working fine except the registry which errors out when trying to read or write content to the minio server.
Here is the debug log of the registry:
time="2019-02-19T10:56:05.020559765Z" level=debug msg="(*linkedBlobStore).Writer" auth.user.name=wzrdtales go.version=go1.7.6 http.request.host="registry.domain.com:443" http.request.id=a4c36bd8-f07f-462f-b67e-f83bf6227c9e http.request.method=POST http.request.remoteaddr= http.request.uri="/v2/test/test/blobs/uploads/" http.request.useragent="docker/18.09.2-ce go/go1.11.5 git-commit/62479626f2 kernel/4.20.8-arch1-1-ARCH os/linux arch/amd64 UpstreamClient(Docker-Client/18.09.2-ce \\(linux\\))" instance.id=8cee6c26-a565-4e2c-9921-b04045f14f3b service=registry vars.name="test/test" version=v2.6.2-14-ga66a4c3
time="2019-02-19T10:56:05.021949136Z" level=debug msg="s3aws.PutContent(\"/docker/registry/v2/repositories/test/test/_uploads/c8edc262-c644-4acb-9cfb-2016a274de0b/startedat\")" auth.user.name=wzrdtales go.version=go1.7.6 http.request.host="registry.domain.com:443" http.request.id=8cf7a6d9-be28-4ed8-8d2f-b1d1d4272bd8 http.request.method=POST http.request.remoteaddr= http.request.uri="/v2/test/test/blobs/uploads/" http.request.useragent="docker/18.09.2-ce go/go1.11.5 git-commit/62479626f2 kernel/4.20.8-arch1-1-ARCH os/linux arch/amd64 UpstreamClient(Docker-Client/18.09.2-ce \\(linux\\))" instance.id=8cee6c26-a565-4e2c-9921-b04045f14f3b service=registry trace.duration=1.413048ms trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).PutContent" trace.id=4e467a29-6ce7-4062-af4b-0d0550177ccb trace.line=95 vars.name="test/test" version=v2.6.2-14-ga66a4c3
time="2019-02-19T10:56:05.022262966Z" level=error msg="response completed with error" auth.user.name=wzrdtales err.code=unknown err.detail="s3aws: SerializationError: failed to decode S3 XML error response" err.message="unknown error" go.version=go1.7.6 http.request.host="registry.domain.com:443" http.request.id=8cf7a6d9-be28-4ed8-8d2f-b1d1d4272bd8 http.request.method=POST http.request.remoteaddr= http.request.uri="/v2/test/test/blobs/uploads/" http.request.useragent="docker/18.09.2-ce go/go1.11.5 git-commit/62479626f2 kernel/4.20.8-arch1-1-ARCH os/linux arch/amd64 UpstreamClient(Docker-Client/18.09.2-ce \\(linux\\))" http.response.contenttype="application/json; charset=utf-8" http.response.duration=10.782219ms http.response.status=500 http.response.written=104 instance.id=8cee6c26-a565-4e2c-9921-b04045f14f3b service=registry vars.name="test/test" version=v2.6.2-14-ga66a4c3
It seems like the registry doesn't support minio, otherwise it wouldn't make sense that it fails even before actually anything is being transferred to the registry. When googling for it, it seems as it should work at least. I tried to check a few things, but fails always with the errors printed above.
The only thing I could think of now is that for some reason it doesn't print a reasonable error and has in the background a problem with the letsencrypt certificate. But it should not of course.