### Summary Cannot push to registry (500 response). ### Steps to reproduce - Restore from backup (omnibus, CentOS or Ubuntu) - Try to push ### Example Project To be honest I did not do backup/restore specifically to verify the reproducibility, but it happened to me twice when restoring for Ubuntu and CentOS. ### What is the current *bug* behavior? Permissions on `/var/opt/gitlab/gitlab-rails/shared/registry` are set to user `git` (not `registry`). Thus, `filesystem: mkdir /var/opt/gitlab/gitlab-rails/shared/registry/docker/registry/v2/repositories/my-repo: permission denied` ### What is the expected *correct* behavior? Successful push. ### Relevant logs and/or screenshots ``` 2019-06-04_22:24:04.76764 time="2019-06-04T18:24:04.767540422-04:00" level=info msg="authorized request" go.version=go1.11.5 http.request.host=registry.dbogatov.org http.request.id=ca82d411-6659-494b-a580-5b61f1bcf6bb http.request.method=POST http.request.remoteaddr=71.235.70.212 http.request.uri="/v2/frankfurt/second-semester/physics-reports/review/master/blobs/uploads/" http.request.useragent="docker/18.06.1-ce go/go1.10.4 git-commit/e68fc7a kernel/4.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/18.06.1-ce \(linux\))" vars.name="frankfurt/second-semester/physics-reports/review/master" 2019-06-04_22:24:04.76799 time="2019-06-04T18:24:04.767862514-04:00" level=error msg="response completed with error" auth.user.name=dbogatov err.code=unknown err.detail="filesystem: mkdir /var/opt/gitlab/gitlab-rails/shared/registry/docker/registry/v2/repositories/frankfurt/second-semester/physics-reports/review/master/_uploads: permission denied" err.message="unknown error" go.version=go1.11.5 http.request.host=registry.dbogatov.org http.request.id=ca82d411-6659-494b-a580-5b61f1bcf6bb http.request.method=POST http.request.remoteaddr=71.235.70.212 http.request.uri="/v2/frankfurt/second-semester/physics-reports/review/master/blobs/uploads/" http.request.useragent="docker/18.06.1-ce go/go1.10.4 git-commit/e68fc7a kernel/4.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/18.06.1-ce \(linux\))" http.response.contenttype="application/json; charset=utf-8" http.response.duration=2.061422ms http.response.status=500 http.response.written=281 vars.name="frankfurt/second-semester/physics-reports/review/master" ``` ### Output of checks #### Results of GitLab environment info <details> <summary>Expand for output related to GitLab environment info</summary> <pre> System information System: CentOS 7.6.1810 Proxy: no Current User: git Using RVM: no Ruby Version: 2.5.3p105 Gem Version: 2.7.9 Bundler Version:1.17.3 Rake Version: 12.3.2 Redis Version: 3.2.12 Git Version: 2.21.0 Sidekiq Version:5.2.7 Go Version: unknown GitLab information Version: 11.11.1-ee Revision: 4ce2f9746d3 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: PostgreSQL DB Version: 9.6.11 URL: https://git.dbogatov.org HTTP Clone URL: https://git.dbogatov.org/some-group/some-project.git SSH Clone URL: ssh://git@git.dbogatov.org:2222/some-group/some-project.git Elasticsearch: yes Geo: no Using LDAP: no Using Omniauth: yes Omniauth Providers: github, bitbucket, gitlab, google_oauth2, facebook, twitter GitLab Shell Version: 9.1.0 Repository storage paths: - default: /var/opt/gitlab/git-data/repositories GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Git: /opt/gitlab/embedded/bin/git </pre> </details> #### Results of GitLab application Check <details> <summary>Expand for output related to the GitLab application check</summary> <pre> Checking GitLab subtasks ... Checking GitLab Shell ... GitLab Shell: ... GitLab Shell version >= 9.1.0 ? ... OK (9.1.0) Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Check GitLab API access: OK Redis available via internal API: OK Access to /var/opt/gitlab/.ssh/authorized_keys: OK gitlab-shell self-check successful Checking GitLab Shell ... Finished Checking Gitaly ... Gitaly: ... default ... OK Checking Gitaly ... Finished Checking Sidekiq ... Sidekiq: ... Running? ... yes Number of Sidekiq processes ... 1 Checking Sidekiq ... Finished Checking Incoming Email ... Incoming Email: ... Reply by email is disabled in config/gitlab.yml Checking Incoming Email ... Finished Checking LDAP ... LDAP: ... LDAP is disabled in config/gitlab.yml Checking LDAP ... Finished Checking GitLab App ... Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... yes Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) Projects have namespace: ... 13/2 ... yes 2/3 ... yes 2/4 ... yes 2/5 ... yes 2/6 ... yes 57/7 ... yes 2/11 ... yes 2/13 ... yes 2/15 ... yes 57/16 ... yes 58/17 ... yes 58/22 ... yes 57/23 ... yes 2/24 ... yes 2/25 ... yes 2/26 ... yes 2/27 ... yes 14/28 ... yes 2/29 ... yes 2/30 ... yes 57/31 ... yes 2/32 ... yes 2/33 ... yes 14/34 ... yes 2/35 ... yes 2/36 ... yes 57/37 ... yes 2/38 ... yes 2/39 ... yes 57/40 ... yes 26/41 ... yes 26/42 ... yes 2/44 ... yes 2/47 ... yes 2/49 ... yes 29/50 ... yes 29/53 ... yes 2/54 ... yes 30/55 ... yes 30/56 ... yes 2/59 ... yes 19/60 ... yes 19/61 ... yes 39/62 ... yes 30/63 ... yes 39/65 ... yes 30/66 ... yes 2/67 ... yes 42/68 ... yes 42/69 ... yes 30/70 ... yes 45/71 ... yes 26/72 ... yes 30/73 ... yes 55/74 ... yes 45/75 ... yes 72/76 ... yes 2/77 ... yes 45/78 ... yes 2/79 ... yes 2/80 ... yes 26/81 ... yes 26/82 ... yes 54/83 ... yes 55/86 ... yes 54/87 ... yes 2/93 ... yes 54/94 ... yes 61/95 ... yes 2/96 ... yes 57/97 ... yes 57/98 ... yes 57/99 ... yes 57/100 ... yes 57/101 ... yes 57/102 ... yes 64/103 ... yes 24/104 ... yes 60/105 ... yes 60/106 ... yes 63/107 ... yes 61/108 ... yes 63/109 ... yes 39/111 ... yes 39/112 ... yes 2/113 ... yes 64/114 ... yes 63/115 ... yes 73/116 ... yes 73/119 ... yes 2/121 ... yes 73/122 ... yes 30/123 ... yes 63/125 ... yes 61/126 ... yes 61/127 ... yes 2/128 ... yes 54/129 ... yes 55/132 ... yes 30/133 ... yes 55/134 ... yes 67/135 ... yes 67/136 ... yes 69/138 ... yes 67/139 ... yes 68/140 ... yes 68/141 ... yes 39/142 ... yes 68/143 ... yes 69/144 ... yes 72/145 ... yes 2/146 ... yes 72/147 ... yes 67/148 ... yes 74/150 ... yes 74/156 ... yes 74/157 ... yes Redis version >= 2.8.0? ... yes Ruby version >= 2.5.3 ? ... yes (2.5.3) Git version >= 2.21.0 ? ... yes (2.21.0) Git user has default SSH configuration? ... yes Active users: ... 21 Elasticsearch version 5.6 - 6.x? ... yes (6.8.0) Checking GitLab App ... Finished Checking GitLab subtasks ... Finished </pre> </details> ### Possible fixes This https://gitlab.com/gitlab-org/gitlab-ce/issues/24250#note_18192290 fixes everything. ### Related I cannot add related issues but - https://gitlab.com/gitlab-org/gitlab-ce/issues/24250 - https://gitlab.com/gitlab-org/gitlab-ce/issues/50785