Blame view

doc/settings/nginx.md 20.8 KB
1 2
# NGINX settings

3 4
## Enable HTTPS

5 6
### Warning

Raphaël Doursenaud committed
7
The Nginx config will tell browsers and clients to only communicate with your
8 9
GitLab instance over a secure connection for the next 24 months. By enabling
HTTPS you'll need to provide a secure connection to your instance for at least
10 11
the next 24 months.

12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
By default, omnibus-gitlab does not use HTTPS. If you want to enable HTTPS for
gitlab.example.com, add the following statement to `/etc/gitlab/gitlab.rb`:

```ruby
# note the 'https' below
external_url "https://gitlab.example.com"
```

Because the hostname in our example is 'gitlab.example.com', omnibus-gitlab
will look for key and certificate files called
`/etc/gitlab/ssl/gitlab.example.com.key` and
`/etc/gitlab/ssl/gitlab.example.com.crt`, respectively. Create the
`/etc/gitlab/ssl` directory and copy your key and certificate there.

```
sudo mkdir -p /etc/gitlab/ssl
sudo chmod 700 /etc/gitlab/ssl
sudo cp gitlab.example.com.key gitlab.example.com.crt /etc/gitlab/ssl/
```

Now run `sudo gitlab-ctl reconfigure`. When the reconfigure finishes your
mryanb committed
33
GitLab instance should be reachable at `https://gitlab.example.com`.
34 35 36 37 38 39 40 41 42 43 44

If you are using a firewall you may have to open port 443 to allow inbound
HTTPS traffic.

```
# UFW example (Debian, Ubuntu)
sudo ufw allow https

# lokkit example (RedHat, CentOS 6)
sudo lokkit -s https

45
# firewall-cmd (RedHat, Centos 7)
46 47 48 49
sudo firewall-cmd --permanent --add-service=https
sudo systemctl reload firewalld
```

50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70
## Redirect `HTTP` requests to `HTTPS`

By default, when you specify an external_url starting with 'https', Nginx will
no longer listen for unencrypted HTTP traffic on port 80. If you want to
redirect all HTTP traffic to HTTPS you can use the `redirect_http_to_https`
setting.

```ruby
external_url "https://gitlab.example.com"
nginx['redirect_http_to_https'] = true
```

## Change the default port and the SSL certificate locations

If you need to use an HTTPS port other than the default (443), just specify it
as part of the external_url.

```ruby
external_url "https://gitlab.example.com:2443"
```

71 72 73
To set the location of ssl certificates create `/etc/gitlab/ssl` directory,
place the `.crt` and `.key` files in the directory and specify the following
configuration:
74 75 76 77 78 79 80

```ruby
# For GitLab
nginx['ssl_certificate'] = "/etc/gitlab/ssl/gitlab.example.crt"
nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/gitlab.example.com.key"
```

81 82
Run `sudo gitlab-ctl reconfigure` for the change to take effect.

83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111
## Change the default proxy headers

By default, when you specify `external_url` omnibus-gitlab will set a few
NGINX proxy headers that are assumed to be sane in most environments.

For example, omnibus-gitlab will set:

```
  "X-Forwarded-Proto" => "https",
  "X-Forwarded-Ssl" => "on"
```

if you have specified `https` schema in the `external_url`.

However, if you have a situation where your GitLab is in a more complex setup
like behind a reverse proxy, you will need to tweak the proxy headers in order
to avoid errors like `The change you wanted was rejected` or
`Can't verify CSRF token authenticity Completed 422 Unprocessable`.

This can be achieved by overriding the default headers, eg. specify
in `/etc/gitlab/gitlab.rb`:

```ruby
 nginx['proxy_set_headers'] = {
  "X-Forwarded-Proto" => "http",
  "CUSTOM_HEADER" => "VALUE"
 }
```

112
Save the file and [reconfigure GitLab](https://docs.gitlab.com/ce/administration/restart_gitlab.html#omnibus-gitlab-reconfigure)
113 114 115 116
for the changes to take effect.

This way you can specify any header supported by NGINX you require.

117
## Configuring GitLab `trusted_proxies` and the NGINX `real_ip` module
118

119
By default, NGINX and GitLab will log the IP address of the connected client.
120 121 122 123 124 125 126 127 128 129

If your GitLab is behind a reverse proxy, you may not want the IP address of
the proxy to show up as the client address.

You can have NGINX look for a different address to use by adding your reverse
proxy to the `real_ip_trusted_addresses` list:

```ruby
# Each address is added to the the NGINX config as 'set_real_ip_from <address>;'
nginx['real_ip_trusted_addresses'] = [ '192.168.1.0/24', '192.168.2.1', '2001:0db8::/32' ]
130
# other real_ip config options
131
nginx['real_ip_header'] = 'X-Real-IP'
132
nginx['real_ip_recursive'] = 'on'
133 134 135 136 137
```

Description of the options:
* http://nginx.org/en/docs/http/ngx_http_realip_module.html

138 139 140 141
By default, omnibus-gitlab will use the IP addresses in `real_ip_trusted_addresses`
as GitLab's trusted proxies, which will keep users from being listed as signed
in from those IPs.

142
Save the file and [reconfigure GitLab](https://docs.gitlab.com/ce/administration/restart_gitlab.html#omnibus-gitlab-reconfigure)
143 144
for the changes to take effect.

145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170
## Configuring HTTP2 protocol

By default, when you specify that your Gitlab instance should be reachable
through HTTPS by specifying `external_url "https://gitlab.example.com"`,
[http2 protocol] is also enabled.

The omnibus-gitlab package sets required ssl_ciphers that are compatible with
http2 protocol.

If you are specifying custom ssl_ciphers in your configuration and a cipher is
in [http2 cipher blacklist], once you try to reach your GitLab instance you will
be presented with `INADEQUATE_SECURITY` error in your browser.

Consider removing the offending ciphers from the cipher list. Changing ciphers
is only necessary if you have a very specific custom setup.

For more info on why you would want to have http2 protocol enabled, check out
the [http2 whitepaper].

If changing the ciphers is not an option you can disable http2 support by
specifying in `/etc/gitlab/gitlab.rb`:

```ruby
nginx['http2_enabled'] = false
```

171
Save the file and [reconfigure GitLab](https://docs.gitlab.com/ce/administration/restart_gitlab.html#omnibus-gitlab-reconfigure)
172 173
for the changes to take effect.

174 175 176 177 178 179 180
## Using a non-bundled web-server

By default, omnibus-gitlab installs GitLab with bundled Nginx.
Omnibus-gitlab allows webserver access through user `gitlab-www` which resides
in the group with the same name. To allow an external webserver access to
GitLab, external webserver user needs to be added `gitlab-www` group.

181 182
To use another web server like Apache or an existing Nginx installation you
will have to perform the following steps:
183

184
1. **Disable bundled Nginx**
185

186
    In `/etc/gitlab/gitlab.rb` set:
187

188 189 190
    ```ruby
    nginx['enable'] = false
    ```
191

192
1. **Set the username of the non-bundled web-server user**
193

194 195 196 197
    By default, omnibus-gitlab has no default setting for the external webserver
    user, you have to specify it in the configuration. For Debian/Ubuntu the
    default user is `www-data` for both Apache/Nginx whereas for RHEL/CentOS
    the Nginx user is `nginx`.
198

199
    *Note: Make sure you have first installed Apache/Nginx so the webserver user is created, otherwise omnibus will fail while reconfiguring.*
200

201 202 203 204 205 206 207 208 209 210 211
    Let's say for example that the webserver user is `www-data`.
    In `/etc/gitlab/gitlab.rb` set:

    ```ruby
    web_server['external_users'] = ['www-data']
    ```

    *Note: This setting is an array so you can specify more than one user to be added to gitlab-www group.*

    Run `sudo gitlab-ctl reconfigure` for the change to take effect.

212
    *Note: if you are using SELinux and your web server runs under a restricted SELinux profile you may have to [loosen the restrictions on your web server][selinuxmod].*
213

214 215
    *Note: make sure that the webserver user has the correct permissions on all directories used by external web-server, otherwise you will receive `failed (XX: Permission denied) while reading upstream` errors.

216 217 218 219 220 221 222 223 224 225 226 227 228
1. **Add the non-bundled web-server to the list of trusted proxies**

    Normally, omnibus-gitlab defaults the list of trusted proxies to the what was
    configured in the real_ip module for the bundled NGINX.

    For non-bundled web-servers the list needs to be configured directly, and should
    include the IP address of your web-server if it not on the same machine as GitLab.
    Otherwise users will be shown as being signed in from your web-server's IP address.

    ```ruby
    gitlab_rails['trusted_proxies'] = [ '192.168.1.0/24', '192.168.2.1', '2001:0db8::/32' ]
    ```

229 230
1. **(Optional) Set the right gitlab-workhorse settings if using Apache**

231 232
    *Note: The values below were added in GitLab 8.2, make sure you have the latest version installed.*

233 234 235 236 237 238
    Apache cannot connect to a UNIX socket but instead needs to connect to a
    TCP Port. To allow gitlab-workhorse to listen on TCP (by default port 8181)
    edit `/etc/gitlab/gitlab.rb`:

    ```
    gitlab_workhorse['listen_network'] = "tcp"
239
    gitlab_workhorse['listen_addr'] = "127.0.0.1:8181"
240
    ```
241

242 243 244 245 246 247 248 249 250 251
    Run `sudo gitlab-ctl reconfigure` for the change to take effect.

1. **Download the right web server configs**

    Go to [GitLab recipes repository][recipes-web] and look for the omnibus
    configs in the webserver directory of your choice. Make sure you pick the
    right configuration file depending whether you choose to serve GitLab with
    SSL or not. The only thing you need to change is `YOUR_SERVER_FQDN` with
    your own FQDN and if you use SSL, the location where your SSL keys currently
    reside. You also might need to change the location of your log files.
252

253 254 255 256 257 258 259 260 261
## Setting the NGINX listen address or addresses

By default NGINX will accept incoming connections on all local IPv4 addresses.
You can change the list of addresses in `/etc/gitlab/gitlab.rb`.

```ruby
nginx['listen_addresses'] = ["0.0.0.0", "[::]"] # listen on all IPv4 and IPv6 addresses
```

262 263 264 265 266
## Setting the NGINX listen port

By default NGINX will listen on the port specified in `external_url` or
implicitly use the right port (80 for HTTP, 443 for HTTPS). If you are running
GitLab behind a reverse proxy, you may want to override the listen port to
267
something else.  For example, to use port 8081:
268 269

```ruby
270
nginx['listen_port'] = 8081
271 272
```

273 274 275 276
## Supporting proxied SSL

By default NGINX will auto-detect whether to use SSL if `external_url`
contains `https://`.  If you are running GitLab behind a reverse proxy, you
277 278 279
may wish to terminate SSL at another proxy server or load balancer. To do this,
be sure the `external_url` contains `https://` and apply the following
configuration to `gitlab.rb`:
280 281

```ruby
282
nginx['listen_port'] = 80
283
nginx['listen_https'] = false
284 285 286 287
nginx['proxy_set_headers'] = {
  "X-Forwarded-Proto" => "https",
  "X-Forwarded-Ssl" => "on"
}
288 289
```

290 291 292 293 294
Note that you may need to configure your reverse proxy or load balancer to
forward certain headers (e.g. `Host`, `X-Forwarded-Ssl`, `X-Forwarded-For`,
`X-Forwarded-Port`) to GitLab. You may see improper redirections or errors
(e.g. "422 Unprocessable Entity", "Can't verify CSRF token authenticity") if
you forget this step. For more information, see:
295

296 297
* http://stackoverflow.com/questions/16042647/whats-the-de-facto-standard-for-a-reverse-proxy-to-tell-the-backend-ssl-is-used
* https://wiki.apache.org/couchdb/Nginx_As_a_Reverse_Proxy
298

299 300 301 302 303 304 305 306
## Setting HTTP Strict Transport Security

By default GitLab enables Strict Transport Security which informs browsers that
they should only contact the website using HTTPS. When browser visits
GitLab instance even once it will remember to no longer attempt insecure connections
even when user is explicitly entering `http://` url. Such url will be automatically redirected by the browser to `https://` variant.

```ruby
307 308
nginx['hsts_max_age'] = 31536000
nginx['hsts_include_subdomains'] = false
309 310 311 312 313 314 315
```

By default `max_age` is set for one year, this is how long browser will remember to only connect through HTTPS. 
Setting `max_age` to 0 will disable this feature. For more information see:

* https://www.nginx.com/blog/http-strict-transport-security-hsts-and-nginx/

316
## Using custom SSL ciphers
317 318 319 320 321 322 323 324 325

By default GitLab is using SSL ciphers that are combination of testing on gitlab.com and various best practices contributed by the GitLab community.

However, you can change the ssl ciphers by adding to `gitlab.rb`:

```ruby
  nginx['ssl_ciphers'] = "CIPHER:CIPHER1"
```

326
and running reconfigure.
327 328 329 330 331 332

You can also enable `ssl_dhparam` directive.

First, generate `dhparams.pem` with `openssl dhparam -out dhparams.pem 2048`. Then, in `gitlab.rb` add a path to the generated file, for example:

```ruby
333
  nginx['ssl_dhparam'] = "/etc/gitlab/ssl/dhparams.pem"
334 335
```

336
After the change run `sudo gitlab-ctl reconfigure`.
337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355

## Enable 2-way SSL Client Authentication

To require web clients to authenticate with a trusted certificate, you can enable 2-way SSL by adding to `gitlab.rb`:

```ruby
  nginx['ssl_verify_client'] = "on"
```

and running reconfigure.

These additional options NGINX supports for configuring SSL client authentication can also be configured:

```ruby
  nginx['ssl_client_certificate'] = "/etc/pki/tls/certs/root-certs.pem"
  nginx['ssl_verify_depth'] = "2"
```

After making the changes run `sudo gitlab-ctl reconfigure`.
356

357 358 359 360 361 362 363 364 365 366 367 368
## Inserting custom NGINX settings into the GitLab server block

If you need to add custom settings into the NGINX `server` block for GitLab for
some reason you can use the following setting.

```ruby
# Example: block raw file downloads from a specific repository
nginx['custom_gitlab_server_config'] = "location ^~ /foo-namespace/bar-project/raw/ {\n deny all;\n}\n"
```

Run `gitlab-ctl reconfigure` to rewrite the NGINX configuration and restart
NGINX.
369

370 371 372 373 374 375 376 377 378 379 380 381 382
## Inserting custom settings into the NGINX config

If you need to add custom settings into the NGINX config, for example to include
existing server blocks, you can use the following setting.

```ruby
# Example: include a directory to scan for additional config files
nginx['custom_nginx_config'] = "include /etc/nginx/conf.d/*.conf;"
```

Run `gitlab-ctl reconfigure` to rewrite the NGINX configuration and restart
NGINX.

383
## Custom error pages
384

385 386 387 388
You can use `custom_error_pages` to modify text on the default GitLab error page.
This can be used for any valid HTTP error code; e.g 404, 502.

As an example the following would modify the default 404 error page.
389 390

```ruby
391
nginx['custom_error_pages'] = {
392 393 394 395 396 397 398 399
  '404' => {
    'title' => 'Example title',
    'header' => 'Example header',
    'message' => 'Example message'
  }
}
```

400 401 402 403
This would result in the 404 error page below.

![custom 404 error page](img/error_page_example.png)

404 405 406
Run `gitlab-ctl reconfigure` to rewrite the NGINX configuration and restart
NGINX.

407 408 409 410 411 412
## Using an existing Passenger/Nginx installation

In some cases you may want to host GitLab using an existing Passenger/Nginx
installation but still have the convenience of updating and installing using
the omnibus packages.

413 414
### Configuration

415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430
First, you'll need to setup your `/etc/gitlab/gitlab.rb` to disable the built-in
Nginx and Unicorn:

```ruby
# Disable the built-in nginx
nginx['enable'] = false

# Disable the built-in unicorn
unicorn['enable'] = false

# Set the internal API URL
gitlab_rails['internal_api_url'] = 'http://git.yourdomain.com'
```

Make sure you run `sudo gitlab-ctl reconfigure` for the changes to take effect.

431 432
### Vhost (server block)

433
Then, in your custom Passenger/Nginx installation, create the following site
434
configuration file:
435 436

```
437 438
upstream gitlab-workhorse {
  server unix://var/opt/gitlab/gitlab-workhorse/socket fail_timeout=0;
439 440
}

441 442
server {
  listen *:80;
443
  server_name git.example.com;
444 445 446 447 448 449 450 451 452 453 454 455
  server_tokens off;
  root /opt/gitlab/embedded/service/gitlab-rails/public;

  client_max_body_size 250m;

  access_log  /var/log/gitlab/nginx/gitlab_access.log;
  error_log   /var/log/gitlab/nginx/gitlab_error.log;

  # Ensure Passenger uses the bundled Ruby version
  passenger_ruby /opt/gitlab/embedded/bin/ruby;

  # Correct the $PATH variable to included packaged executables
456
  passenger_env_var PATH "/opt/gitlab/bin:/opt/gitlab/embedded/bin:/usr/local/bin:/usr/bin:/bin";
457 458 459 460 461 462 463 464 465 466

  # Make sure Passenger runs as the correct user and group to
  # prevent permission issues
  passenger_user git;
  passenger_group git;

  # Enable Passenger & keep at least one instance running at all times
  passenger_enabled on;
  passenger_min_instances 1;

467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500
  location ~ ^/[\w\.-]+/[\w\.-]+/(info/refs|git-upload-pack|git-receive-pack)$ {
    # 'Error' 418 is a hack to re-use the @gitlab-workhorse block
    error_page 418 = @gitlab-workhorse;
    return 418;
  }

  location ~ ^/[\w\.-]+/[\w\.-]+/repository/archive {
    # 'Error' 418 is a hack to re-use the @gitlab-workhorse block
    error_page 418 = @gitlab-workhorse;
    return 418;
  }

  location ~ ^/api/v3/projects/.*/repository/archive {
    # 'Error' 418 is a hack to re-use the @gitlab-workhorse block
    error_page 418 = @gitlab-workhorse;
    return 418;
  }

  # Build artifacts should be submitted to this location
  location ~ ^/[\w\.-]+/[\w\.-]+/builds/download {
      client_max_body_size 0;
      # 'Error' 418 is a hack to re-use the @gitlab-workhorse block
      error_page 418 = @gitlab-workhorse;
      return 418;
  }

  # Build artifacts should be submitted to this location
  location ~ /ci/api/v1/builds/[0-9]+/artifacts {
      client_max_body_size 0;
      # 'Error' 418 is a hack to re-use the @gitlab-workhorse block
      error_page 418 = @gitlab-workhorse;
      return 418;
  }

501
  # For protocol upgrades from HTTP/1.0 to HTTP/1.1 we need to provide Host header if its missing
502
  if ($http_host = "") {
503 504
  # use one of values defined in server_name
    set $http_host_with_default "git.example.com";
505 506 507 508 509 510
  }

  if ($http_host != "") {
    set $http_host_with_default $http_host;
  }

511
  location @gitlab-workhorse {
512 513 514

    ## https://github.com/gitlabhq/gitlabhq/issues/694
    ## Some requests take more than 30 seconds.
515
    proxy_read_timeout      3600;
516 517 518 519 520 521
    proxy_connect_timeout   300;
    proxy_redirect          off;

    # Do not buffer Git HTTP responses
    proxy_buffering off;

522
    proxy_set_header    Host                $http_host_with_default;
523 524 525
    proxy_set_header    X-Real-IP           $remote_addr;
    proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
    proxy_set_header    X-Forwarded-Proto   $scheme;
526

527
    proxy_pass http://gitlab-workhorse;
528

529 530 531 532 533
    ## The following settings only work with NGINX 1.7.11 or newer
    #
    ## Pass chunked request bodies to gitlab-workhorse as-is
    # proxy_request_buffering off;
    # proxy_http_version 1.1;
534 535
  }

536 537 538 539 540 541 542 543 544 545
  ## Enable gzip compression as per rails guide:
  ## http://guides.rubyonrails.org/asset_pipeline.html#gzip-compression
  ## WARNING: If you are using relative urls remove the block below
  ## See config/application.rb under "Relative url support" for the list of
  ## other files that need to be changed for relative url support
  location ~ ^/(assets)/ {
    root /opt/gitlab/embedded/service/gitlab-rails/public;
    gzip_static on; # to serve pre-gzipped version
    expires max;
    add_header Cache-Control public;
546 547
  }

548
  error_page 502 /502.html;
549 550 551
}
```

552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574
### Enabling/Disabling nginx_status

By default you will have an nginx health-check endpoint configured at 127.0.0.1:8060/nginx_status to monitor your Nginx server status.

#### The following information will be displayed:

```
Active connections: 1
server accepts handled requests
 18 18 36
Reading: 0 Writing: 1 Waiting: 0
```
* Active connections – Open connections in total.
* 3 figures are shown.
 * All accepted connections.
 * All handled connections.
 * Total number of handled requests.
* Reading: Nginx reads request headers
* Writing: Nginx reads request bodies, processes requests, or writes responses to a client
* Waiting: Keep-alive connections. This number depends on the keepalive-timeout.

## Configuration

575
Edit `/etc/gitlab/gitlab.rb`:
576 577

```Ruby
578 579 580 581 582
nginx['status'] = {
  "listen_addresses" => ["127.0.0.1"],
  "fqdn" => "dev.example.com",
  "port" => 9999,
  "options" => {
583
    "stub_status" => "on", # Turn on stats
584
    "access_log" => "on", # Disable logs for stats
585
    "allow" => "127.0.0.1", # Only allow access from localhost
586 587
    "deny" => "all" # Deny access to anyone else
  }
588 589 590 591 592 593
}
```

If you don't find this service useful for your current infrastructure you can disable it with:

```ruby
594 595 596
nginx['status'] = {
  'enable' => false
}
597 598 599 600 601 602
```

Make sure you run sudo gitlab-ctl reconfigure for the changes to take effect.



603 604
#### Warning

605 606 607 608 609 610 611
To ensure that user uploads are accessible your Nginx user (usually `www-data`)
should be added to the `gitlab-www` group. This can be done using the following command:

```shell
sudo usermod -aG gitlab-www www-data
```

612 613
#### Templates

614
Other than the Passenger configuration in place of Unicorn and the lack of HTTPS
615 616
(although this could be enabled) these files are mostly identical to :

617
- [bundled Gitlab Nginx configuration][nginx-cookbook]
618 619

Don't forget to restart Nginx to load the new configuration (on Debian-based
620 621 622
systems `sudo service nginx restart`).

[recipes-web]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server
623
[selinuxmod]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache#selinux-modifications
624 625 626
[http2 protocol]: https://tools.ietf.org/html/rfc7540
[http2 whitepaper]: https://assets.wp.nginx.com/wp-content/uploads/2015/09/NGINX_HTTP2_White_Paper_v4.pdf?_ga=1.127086286.212780517.1454411744
[http2 cipher blacklist]: https://tools.ietf.org/html/rfc7540#appendix-A
627
[nginx-cookbook]: https://gitlab.com/gitlab-org/omnibus-gitlab/tree/master/files/gitlab-cookbooks/gitlab/templates/default/nginx-gitlab-http.conf.erb