Built-in NGINX does not get SSL crt updates on a gitlab-ctl reconfigure

Summary

NGINX does not update certificates after a gitlab-ctl reconfigure, if there was a functional certificate already there.

Steps to reproduce

(Mentions of the 'NGINX' server in this report are reffering to the 'baked in' or included NGINX server that comes with gitlab CE.)

  1. Configure Gitlab for SSL
  2. Place the SSL key in the appropriate .key file
  3. Place JUST the server certificate in the appropriate .crt file
  4. Run gitlab-ctl reconfigure
  5. Run openssl s_client -showcerts -connect your.hostname.net:443 AND keep the results handy

THEN

  1. Add the intermediate certificates to the appropriate .crt file
  2. Run gitlab-ctl reconfigure
  3. Run openssl s_client -showcerts -connect your.hostname.net:443 and compare the results to the earlier run.
  4. Reboot the server (or restart the NGINX server manually)
  5. Run openssl s_client -showcerts -connect your.hostname.net:443 NOW the updates to the .crt file are observed in the NGINX server.

What is the current bug behavior?

Changes to the .crt file are not reflected in the NGINX server after a gitlab-ctl reconfigure

What is the expected correct behavior?

Changes to the .crt file should update the NGINX server after a gitlab-ctl reconfigure, not requiring a server restart, or other service restart.

Relevant logs and/or screenshots

The changes to the .crt file are not being detected, therefore they are not being logged.

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info 
System information
System:		Ubuntu 16.04
Current User:	git
Using RVM:	no
Ruby Version:	2.3.5p376
Gem Version:	2.6.6
Bundler Version:1.13.7
Rake Version:	12.0.0
Redis Version:	3.2.5
Git Version:	2.13.5
Sidekiq Version:5.0.4
Go Version:	unknown


GitLab information
Version:	9.5.5
Revision:	3cf5eba
Directory:	/opt/gitlab/embedded/service/gitlab-rails
DB Adapter:	postgresql
URL:		https://gitlab.mycorp.net
HTTP Clone URL:	https://gitlab.mycorp.net/some-group/some-project.git
SSH Clone URL:	git@gitlab.mycorp.net:some-group/some-project.git
Using LDAP:	yes
Using Omniauth:	no


GitLab Shell
Version:	5.8.0
Repository storage paths:
  • default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check 
Checking GitLab Shell ...


GitLab Shell version >= 5.8.0 ? ... OK (5.8.0)
Repo base directory exists?
default... yes
Repo storage directories are symlinks?
default... no
Repo paths owned by git:root, or git:git?
default... yes
Repo paths access is drwxrws---?
default... yes
hooks directories in repos are links: ...
2/1 ... ok
2/3 ... ok
6/4 ... ok
2/6 ... ok
2/7 ... ok
2/8 ... repository is empty
2/9 ... ok
8/10 ... ok
2/11 ... ok
11/12 ... ok
9/13 ... ok
2/14 ... ok
2/15 ... ok
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Check GitLab API access: OK
Access to /var/opt/gitlab/.ssh/authorized_keys: OK
Send ping to redis server: OK
gitlab-shell self-check successful


Checking GitLab Shell ... Finished


Checking Sidekiq ...


Running? ... yes
Number of Sidekiq processes ... 1


Checking Sidekiq ... Finished


Checking Reply by email ...


Reply by email is disabled in config/gitlab.yml


Checking Reply by email ... Finished


Checking LDAP ...


Server: ldapmain
LDAP authentication... Success
LDAP users with access to your GitLab server (only showing the first 100 results)
HAHA, no. I'm not sharing my LDAP users and structure outside of my org :)
Also, the issue is not related to LDAP.


Checking LDAP ... Finished


Checking GitLab ...


Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
2/1 ... yes
2/3 ... yes
6/4 ... yes
2/6 ... yes
2/7 ... yes
2/8 ... yes
2/9 ... yes
8/10 ... yes
2/11 ... yes
11/12 ... yes
9/13 ... yes
2/14 ... yes
2/15 ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.3.3 ? ... yes (2.3.5)
Git version >= 2.7.3 ? ... yes (2.13.5)
Active users: ... 12


Checking GitLab ... Finished

Possible fixes

I'm not familiar with Chef, but I suspect a check of the last modified time of the .crt and .key files, compared to the last time gitlab-ctl reconfigure was ran, would be enough to determine 'hey, we should restart NGINX because these files were changed'. I'd think something along those lines would resolve this issue.