Docker registry doesn't work with `Host` setting with standard port
As reported in https://gitlab.com/gitlab-org/gitlab-ce/issues/20185, registry.gitlab.com is failing today after the upgrade to 8.10.1:
$ docker push registry.gitlab.com/stanhu/gitlab-ce
The push refers to a repository [registry.gitlab.com/stanhu/gitlab-ce]
2846b84962b1: Pushing [==================================================>] 1.536 kB
a2adbfc2ba5a: Pushing [==================================================>] 6.656 kB
989da4e4d422: Pushing [==================================================>] 5.12 kB
c6f3980fc407: Pushing [> ] 544.8 kB/216.8 MB
a8ea8e8dd3fa: Pushing [==================================================>] 3.584 kB
7f5dcbf5b81f: Waiting
6905deb57b14: Waiting
4ee4d807cfd9: Waiting
0e6048bd4126: Waiting
04e643fddbab: Waiting
42755cf4ee95: Waiting
unauthorized: authentication requiredI looked at the debug logs of my daemon:
time="2016-07-25T14:10:14.305594972-04:00" level=debug msg="Calling POST /v1.23/images/registry.gitlab.com/stanhu/gitlab-ce/push?tag="
time="2016-07-25T14:10:14.339862033-04:00" level=debug msg="hostDir: /etc/docker/certs.d/registry.gitlab.com"
time="2016-07-25T14:10:14.357431427-04:00" level=debug msg="hostDir: /etc/docker/certs.d/registry.gitlab.com"
time="2016-07-25T14:10:14.358030170-04:00" level=debug msg="Trying to push registry.gitlab.com/stanhu/gitlab-ce to https://registry.gitlab.com v2"
time="2016-07-25T14:10:14.459248848-04:00" level=debug msg="Pushing repository: registry.gitlab.com/stanhu/gitlab-ce:latest"
time="2016-07-25T14:10:14.460723272-04:00" level=debug msg="Pushing layer: sha256:a8ea8e8dd3fae4dadafbeb44bda5f7c7a43362cddfddc74cc60435c6d18362af"
time="2016-07-25T14:10:14.462155913-04:00" level=debug msg="Pushing layer: sha256:2846b84962b1e3837df54d5475115723b1d52f78b36d4e1b733a346092ddce8a"
time="2016-07-25T14:10:14.462721617-04:00" level=debug msg="Pushing layer: sha256:a2adbfc2ba5a956812e5844f18ed4bfd07cfe27894bfd356cfd349a9cfbdceb9"
time="2016-07-25T14:10:14.463257936-04:00" level=debug msg="Pushing layer: sha256:989da4e4d4220af2b0556cbbf2c820d41f83f90aba9a1de7f46800ea929f5b39"
time="2016-07-25T14:10:14.463775204-04:00" level=debug msg="Pushing layer: sha256:c6f3980fc407e1a11e6ecf25c1b134713bdbdbc47ffea26a769b8e45f3fe8141"
time="2016-07-25T14:10:14.682108328-04:00" level=debug msg="Increasing token expiration to: 60 seconds"
time="2016-07-25T14:10:14.819539168-04:00" level=debug msg="Assembling tar data for ca51b7f0ff63cf37e4fd997d603aad3e8f32cf299599cfa3f9ce63a77cfb0244"
time="2016-07-25T14:10:14.838907752-04:00" level=debug msg="Assembling tar data for 8e8fe3245f40d8b8a32893afd962052a26ececcff81bc11529747f598a986626"
time="2016-07-25T14:10:14.854679681-04:00" level=debug msg="Assembling tar data for 87ae72769a662b983f708bf54c6a0fba5c47c8f80ad2233475e967ffe6b1083c"
time="2016-07-25T14:10:14.857028479-04:00" level=debug msg="Assembling tar data for b0dc602f807a2c1a1ac29daba1bd773259e0622cfa4f3e179d6061d6866fa20f"
time="2016-07-25T14:10:14.861980017-04:00" level=debug msg="Assembling tar data for ee1f2259af7980e52a1a3362d21b7cec2aed2ff18bc4f774e1dc505a164f682a"
time="2016-07-25T14:10:14.967268545-04:00" level=error msg="Upload failed: unauthorized: authentication required"
time="2016-07-25T14:10:14.967567405-04:00" level=debug msg="Pushing layer: sha256:7f5dcbf5b81fa451830d958845d84f9da11a6e6343cf73ab5c38752ad7479e8f"
time="2016-07-25T14:10:14.968417701-04:00" level=error msg="Upload failed: unauthorized: authentication required"
time="2016-07-25T14:10:14.968576959-04:00" level=debug msg="Pushing layer: sha256:6905deb57b146b0bc85ed8baa326ae3678d77dc4c7436a46811b172045d028f5"
time="2016-07-25T14:10:14.968910647-04:00" level=error msg="Attempting next endpoint for push after error: unauthorized: authentication required"
time="2016-07-25T14:10:14.968980351-04:00" level=debug msg="Skipping v1 endpoint https://registry.gitlab.com because v2 registry was detected" @ahanselka was able to reproduce this issue by using a reverse proxy in front of the GitLab server. He was able to resolve the issue by adding the lines in /etc/gitlab/gitlab.rb:
registry_nginx['proxy_set_headers'] = { "Host" => "registry.mydomain.com" }Then running sudo gitlab-ctl reconfigure.
According to this issue in the Docker distribution, we shouldn't set the Host header to include port 443 unless it's a non-standard port. I'm not sure why this is the case. This seems to work fine on my local instance without a reverse proxy in front of GitLab.