Unable to Connect to Gitlab Instance's ngnix (Connection refused, ngnix down, ver 11.8.0)

Summary

Trying to navigate to a gitlab instance is meet with a ERR_CONNECTION_REFUSED error.

Steps to reproduce

This one is hard, I only noticed it today when jumping on to check out a CI failure. The only odd thing was the dir for letencypt keys was changed on the host, and gitlab couldn't access them. I've rectified that issue. However, the docker image in use is the latest one.

Edit: I restored my data folders from a backup and reverted to version "11.7.5-ce.0". The issue is not present there.

What is the current bug behavior?

Trying to navigate to a gitlab instance is meet with a ERR_CONNECTION_REFUSED error. nginx is down and unable to be started

What is the expected correct behavior?

The instances webpage.

Relevant logs and/or screenshots

Further testing from within the docker image with curl also returns a connection refused. Running gitlab-ctl status show that ngnix is down, and attempts to restart it fail.

# curl localhost
curl: (7) Failed to connect to localhost port 80: Connection refused
# curl localhost:9080
curl: (7) Failed to connect to localhost port 9080: Connection refused
# gitlab-ctl status
run: alertmanager: (pid 1330) 3299s; run: log: (pid 1113) 3322s
run: gitaly: (pid 1255) 3301s; run: log: (pid 819) 3341s
run: gitlab-monitor: (pid 916) 3336s; run: log: (pid 967) 3335s
run: gitlab-workhorse: (pid 1221) 3301s; run: log: (pid 713) 3361s
run: logrotate: (pid 734) 3354s; run: log: (pid 745) 3353s
down: nginx: 0s, normally up, want up; run: log: (pid 720) 3359s
run: postgres-exporter: (pid 1349) 3299s; run: log: (pid 1158) 3319s
run: postgresql: (pid 493) 3376s; run: log: (pid 638) 3375s
run: prometheus: (pid 1307) 3300s; run: log: (pid 1103) 3323s
run: redis: (pid 452) 3377s; run: log: (pid 494) 3376s
run: redis-exporter: (pid 1274) 3300s; run: log: (pid 1051) 3328s
run: registry: (pid 1239) 3301s; run: log: (pid 772) 3347s
run: sidekiq: (pid 8169) 1532s; run: log: (pid 693) 3366s
run: sshd: (pid 23) 3393s; run: log: (pid 22) 3393s
run: unicorn: (pid 8129) 1535s; run: log: (pid 681) 3369s
# gitlab-ctl start nginx
timeout: down: nginx: 1s, normally up, want up
#

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info 
System information
System:
Current User:   git
Using RVM:      no
Ruby Version:   2.5.3p105
Gem Version:    2.7.6
Bundler Version:1.16.6
Rake Version:   12.3.2
Redis Version:  3.2.12
Git Version:    2.18.1
Sidekiq Version:5.2.5
Go Version:     unknown

GitLab information
Version:        11.8.0
Revision:       ce13864
Directory:      /opt/gitlab/embedded/service/gitlab-rails
DB Adapter:     postgresql
URL:            https://codelabs.alteria.xyz
HTTP Clone URL: https://codelabs.alteria.xyz/some-group/some-project.git
SSH Clone URL:  git@codelabs.alteria.xyz:some-group/some-project.git
Using LDAP:     no
Using Omniauth: yes
Omniauth Providers: google_oauth2, github, AlteriaAuth


GitLab Shell
Version:        8.4.4
Repository storage paths:
  • default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check 
Checking GitLab subtasks ...


Checking GitLab Shell ...


GitLab Shell: ... GitLab Shell version >= 8.4.4 ? ... OK (8.4.4)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Check GitLab API access: OK
Redis available via internal API: OK


Access to /var/opt/gitlab/.ssh/authorized_keys: OK
gitlab-shell self-check successful


Checking GitLab Shell ... Finished


Checking Gitaly ...


Gitaly: ... default ... OK


Checking Gitaly ... Finished


Checking Sidekiq ...


Sidekiq: ... Running? ... yes
Number of Sidekiq processes ... 1


Checking Sidekiq ... Finished


Checking Incoming Email ...


Incoming Email: ... Reply by email is disabled in config/gitlab.yml


Checking Incoming Email ... Finished


Checking LDAP ...


LDAP: ... LDAP is disabled in config/gitlab.yml


Checking LDAP ... Finished


Checking GitLab App ...


Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
2/1 ... yes
2/2 ... yes
2/3 ... yes
2/4 ... yes
2/5 ... yes
2/9 ... yes
2/10 ... yes
2/12 ... yes
2/13 ... yes
2/16 ... yes
2/17 ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.3.5 ? ... yes (2.5.3)
Git version >= 2.18.0 ? ... yes (2.18.1)
Git user has default SSH configuration? ... yes
Active users: ... 4


Checking GitLab App ... Finished


Checking GitLab subtasks ... Finished

Possible fixes

(If you can, link to the line of code that might be responsible for the problem)

Edited by Ryonez