Gitlab Pages throw 502 error in Docker environment after update from 13.12 to 14.02

Summary

Yesterday night, we updated our Dockerized Gitlab CE environment to 14.02.
After the update to 14.01, the pages are gone and throw an error 502. We then updated to 14.02 to check, if there is an fix included, but then stop updating and check the reason.
There is also an entry within the forun: https://forum.gitlab.com/t/pages-error-502/58477

What is the current bug behavior?

Pages show 502 error

What is the expected correct behavior?

Pages show page content

Relevant logs and/or screenshots

/var/log/gitlab/gitlab-pages/current

{"correlation_id":"01FG7VMKWBA7TR8P72XD3JSZCG","error":null,"level":"debug","lookup_error":"Get \"https://git.redoo-networks.com/api/v4/internal/pages?host=custompage.domain.com\": dial tcp 192.168.48.16:443: c
onnect: connection refused","lookup_name":"custompage.domain.com","lookup_paths":null,"msg":"retrieval response sent","requested_domain":"custompage.domain.com","time":"2021-09-22T22:57:23Z"}

{"correlation_id":"01FG7VMKWBA7TR8P72XD3JSZCG","error":"Get \"https://git.redoo-networks.com/api/v4/internal/pages?host=custompage.domain.com\": dial tcp 192.168.48.16:443: connect: connection refused","host":"
custompage.domain.com","level":"error","msg":"could not fetch domain information from a source","path":"/app/sessions/signIn","time":"2021-09-22T22:57:23Z"}

Results of GitLab environment info

We use dockerized Gitlab behind a Traefik reverse proxy, including SSL Termination. The gitlab docker environment that's why is configured with "nginx['listen_https'] = false", like you recommend here: https://docs.gitlab.com/omnibus/settings/nginx.html#external-proxy-and-load-balancer-ssl-termination

Expand for output related to GitLab environment info

System information
System:
Current User:   git
Using RVM:      no
Ruby Version:   2.7.2p137
Gem Version:    3.1.4
Bundler Version:2.1.4
Rake Version:   13.0.6
Redis Version:  6.0.14
Git Version:    2.32.0
Sidekiq Version:5.2.9
Go Version:     unknown

GitLab information
Version:        14.2.4
Revision:       25bed1192b4
Directory:      /opt/gitlab/embedded/service/gitlab-rails
DB Adapter:     PostgreSQL
DB Version:     12.8
URL:            https://git.redoo-networks.com
HTTP Clone URL: https://git.redoo-networks.com/some-group/some-project.git
SSH Clone URL:  ssh://git@git.redoo-networks.com:2233/some-group/some-project.git
Using LDAP:     yes
Using Omniauth: yes
Omniauth Providers: 

GitLab Shell
Version:        13.19.1
Repository storage paths:
- default:      /var/opt/gitlab/git-data/repositories
GitLab Shell path:              /opt/gitlab/embedded/service/gitlab-shell
Git:            /opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check
Checking GitLab subtasks ...

Checking GitLab Shell ...

GitLab Shell: ... GitLab Shell version >= 13.19.1 ? ... OK (13.19.1) Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Internal API available: OK Redis available via internal API: OK gitlab-shell self-check successful

Checking GitLab Shell ... Finished

Checking Gitaly ...

Gitaly: ... default ... OK

Checking Gitaly ... Finished

Checking Sidekiq ...

Sidekiq: ... Running? ... yes Number of Sidekiq processes (cluster/worker) ... 1/1

Checking Sidekiq ... Finished

Checking Incoming Email ...

Incoming Email: ... Reply by email is disabled in config/gitlab.yml

Checking Incoming Email ... Finished

Checking LDAP ...

LDAP: ... Server: ldapmain LDAP authentication... Success LDAP users with access to your GitLab server (only showing the first 100 results) User output sanitized. Found 20 users of 100 limit.

Checking LDAP ... Finished

Possible fixes

Because https is disabled in nginx, he only listen on port 80, like you propose in docs.
But within the gitlab docker container, you, or the based ubuntu image, pin the local ip to hostname within /etc/hosts:
192.168.48.16 git.redoo-networks.com git

When the gitlab-pages daemon now tries to contact "https://git.redoo-networks.com/...", he get a "Connected refused", because local IP is only listing on http port 80. We found a quick workaround to modify /etc/hosts and set our gitlab hostname to the IP of traefik proxy. (192.168.48.13) So the request was possible and pages come back. But this is only a temporarily solution, because after restart the ip is set back to normal/internal one.

We will enable nginx['listen_https'] during next maintenance window next night to check, if this solves the problem permanently.
But then I would recommend, you adjust your docs.
Another possible fix also would be to configure gitlab-pages to do requests to gitlab over http (but insecure)

Assignee Loading
Time tracking Loading