Skip to content

External Nginx doesn't integrate seamlessly on SeLinux (CentOS) hosts

Summary

[Self-Managed]

When setting up an external Nginx on a SeLinux system, it is necessary to change the security context of the workhorse's sockets in order to allow the external Nginx to access that socket.

This action has to be done after every execution of gitlab-ctl reconfigure.

Steps to reproduce

  • Spin a GitLab instance on a SeLinux (or CentOS) host,
  • disable embedded Nginx
  • install and configure an external Nginx (following our docs: Using a non-bundled web server)
  • restart the external nginx; reconfigure GitLab
  • Observe a 502: Bad gateway error when accessing the GitLab UI

As per https://stackoverflow.com/a/50052893

This can fixed by applying chcon -t httpd_var_run_t /var/opt/gitlab/gitlab-workhorse/sockets/socket (each-time after executing gitlab-ctl reconfigure )

That approach was recently confirmed during a customer emergency call (internal link to ZD ticket)

Example Project

What is the current bug behavior?

The Workhorse's socket file is inaccessible for the non-bundled Nginx.

What is the expected correct behavior?

The need of running any additional commands after each gitlab-ctl reconfigure causes manual overhead and is prone to errors.

A GitLab instance should be able to handle even such "edge-configurations" automatically.

Relevant logs and/or screenshots

In the Nginx logs following message can be seen (while seeing a 500 error when accessing the GitLab UI):

<TIMESTAMP> [crit]
15777#15777: *1333 connect() to unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket failed (13:
Permission denied) while connecting to upstream, client: <IP-ADDRESS>,
server: <SERVER-NAME>, request: $uest: "GET //sito/wp-includes/wlwmanifest.xml HTTP/1.1", 
upstream: "http://unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket://sito/wp-includes/wlwmanifest.xml",
host: "<HOST-NAME"

Workhorse doesn't show/report any errors, as the external Nginx never reaches its socket due to the SeLinux restrictions.

Output of checks

Results of GitLab environment info

GitLab version: 14.1.3-ee
Ruby: 2.7.2pl37
Gem: 3.1.4
Bundler: 2.1.4
Rake: 13.0.3
Redis: 6.0.14
Git: 2.32.0
Sidekiq: 5.2.9
Expand for output related to GitLab environment info 

(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

Results of GitLab application Check

Expand for output related to the GitLab application check 
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)


(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)


(we will only investigate if the tests are passing)

Possible fixes

Edited by Vladimír Brodský