Skip to content

Socket permission issue when using internal nginx and gitlab/gitlab-ce docker image

After updating docker recently, my gitlab image was restarted and it all stopped working and I get a 502 error. I use the bundled nginx web server.

I restarted the gitlab image and inside of the image did a gitlab-ctl reconfigure followed by a gitlab-ctl restart.

In /var/log/gitlab/nginx/gitlab_error.log I find log entries like these:

2017/04/26 13:58:00 [crit] 484#0: *4173 connect() to unix:/var/opt/gitlab/gitlab-workhorse/socket failed (13: Permission denied) while connecting to upstream, client: 172.17.0.4, server: gitlab.mydomain.cloud, request: "POST /api/v4/jobs/request HTTP/1.1", upstream: "http://unix:/var/opt/gitlab/gitlab-workhorse/socket:/api/v4/jobs/request", host: "gitlab.mydomain.cloud"
2017/04/26 13:58:00 [crit] 484#0: *4175 connect() to unix:/var/opt/gitlab/gitlab-workhorse/socket failed (13: Permission denied) while connecting to upstream, client: 172.17.0.4, server: gitlab.mydomain.cloud, request: "POST /api/v4/runners/verify HTTP/1.1", upstream: "http://unix:/var/opt/gitlab/gitlab-workhorse/socket:/api/v4/runners/verify", host: "gitlab.mydomain.cloud"

This suggests, that nginx cannot access the workhorse socket, similar to gitlab-workhorse#26 (closed) (but: we are using the bundled nginx web server).

The socket file has the following permissions:

srwxr-x---+ 1 git git 0 Apr 26 13:45 /var/opt/gitlab/gitlab-workhorse/socket

The integrated nginx web server seems to run as gitlab-www which has no access to that socket as far as I can tell. From the other thread I see, that it should have permissions srwxrwxrwx, is that still correct? If so, why does it not have those permissions assigned?

Here is my gitlab.rb config file:

external_url 'https://gitlab.mydomain.cloud'
gitlab_rails['gitlab_shell_ssh_port'] = 2222
gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "*******"
gitlab_rails['smtp_port'] = 587
gitlab_rails['smtp_user_name'] = "*******"
gitlab_rails['smtp_password'] = "*******"
gitlab_rails['smtp_domain'] = "******"
gitlab_rails['smtp_authentication'] = "login"
gitlab_rails['smtp_enable_starttls_auto'] = true
gitlab_rails['smtp_tls'] = false
gitlab_rails['smtp_openssl_verify_mode'] = 'none'
gitlab_shell['http_settings'] = { ca_file: '/etc/gitlab/certs/mydomain.cloud.pem', ca_path: '/etc/gitlab/certs', self_signed_cert: false}
nginx['listen_port'] = 80
nginx['listen_https'] = false
nginx['proxy_set_headers'] = {
  "X-Forwarded-Proto" => "https",
  "X-Forwarded-Ssl" => "on"
}
ci_external_url 'http://gitlab-ci.mydomain.cloud'

Note, that I have a https external url and yet do not listen on https, because the gitlab-ce docker container is hidden behind another nginx reverse proxy docker container, which does the https.

Edited by 🤖 GitLab Bot 🤖