error: RPC failed; HTTP 502 curl 22 The requested URL returned error: 502 Bad Gateway fatal: The remote end hung up unexpectedly

Summary

error: RPC failed; HTTP 502 curl 22 The requested URL returned error: 502 Bad Gateway fatal: The remote end hung up unexpectedly after upgrade from 8.15.4 to 8.16.0/8.16.1/8.16.2/8.16.3

Steps to reproduce

Upgrade from 8.15.4 to 8.16.0/8.16.1/8.16.2/8.16.3 and perform gitlab-ctl pg-upgrade

Expected behavior

http authentication should work and resolve to repo's.

Actual behavior

http authentication doesn't go through throws below message error: RPC failed; HTTP 502 curl 22 The requested URL returned error: 502 Bad Gateway fatal: The remote end hung up unexpectedly

Relevant logs and/or screenshots

error: RPC failed; HTTP 502 curl 22 The requested URL returned error: 502 Bad Gateway fatal: The remote end hung up unexpectedly

Output of checks

(If you are reporting a bug on GitLab.com, write: This bug happens on GitLab.com)

Results of GitLab application Check

Checking GitLab Shell ...

GitLab Shell version >= 4.1.1 ? ... OK (4.1.1) Repo base directory exists? default... yes Repo storage directories are symlinks? default... no Repo paths owned by git:git? default... yes Repo paths access is drwxrws---? default... yes hooks directories in repos are links: ... 1/40 ... ok 1/41 ... ok 1/42 ... ok 1/43 ... ok 1/44 ... ok 1/45 ... ok 1/47 ... ok 1/49 ... ok 1/50 ... ok 1/51 ... ok 1/53 ... ok 1/54 ... ok 1/55 ... ok 1/56 ... ok 1/57 ... ok 1/59 ... ok 1/61 ... ok 1/62 ... ok 1/63 ... ok 1/64 ... ok 1/65 ... ok 1/66 ... ok 1/67 ... ok 1/68 ... ok 1/71 ... ok 1/72 ... ok 1/73 ... ok 1/74 ... ok 1/77 ... ok 1/78 ... ok 1/79 ... ok 1/80 ... ok 1/83 ... ok 1/84 ... ok 1/85 ... ok 1/86 ... ok 1/87 ... ok 35/89 ... ok 1/90 ... ok 13/92 ... ok 1/93 ... ok 1/95 ... repository is empty 1/96 ... ok 1/97 ... ok 1/98 ... ok 1/109 ... ok 1/110 ... ok 1/111 ... repository is empty 1/112 ... repository is empty Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Check GitLab API access: OK Access to /var/opt/gitlab/.ssh/authorized_keys: OK Send ping to redis server: OK gitlab-shell self-check successful

Checking GitLab Shell ... Finished

Checking Sidekiq ...

Running? ... yes Number of Sidekiq processes ... 1

Checking Sidekiq ... Finished

Checking Reply by email ...

Reply by email is disabled in config/gitlab.yml

Checking Reply by email ... Finished

Checking LDAP ...

LDAP is disabled in config/gitlab.yml

Checking LDAP ... Finished

Checking GitLab ...

Git configured with autocrlf=input? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config outdated? ... no Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory setup correctly? ... yes Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) projects have namespace: ... 1/40 ... yes 1/41 ... yes 1/42 ... yes 1/43 ... yes 1/44 ... yes 1/45 ... yes 1/47 ... yes 1/49 ... yes 1/50 ... yes 1/51 ... yes 1/53 ... yes 1/54 ... yes 1/55 ... yes 1/56 ... yes 1/57 ... yes 1/59 ... yes 1/61 ... yes 1/62 ... yes 1/63 ... yes 1/64 ... yes 1/65 ... yes 1/66 ... yes 1/67 ... yes 1/68 ... yes 1/71 ... yes 1/72 ... yes 1/73 ... yes 1/74 ... yes 1/77 ... yes 1/78 ... yes 1/79 ... yes 1/80 ... yes 1/83 ... yes 1/84 ... yes 1/85 ... yes 1/86 ... yes 1/87 ... yes 35/89 ... yes 1/90 ... yes 13/92 ... yes 1/93 ... yes 1/95 ... yes 1/96 ... yes 1/97 ... yes 1/98 ... yes 1/109 ... yes 1/110 ... yes 1/111 ... yes 1/112 ... yes Redis version >= 2.8.0? ... yes Ruby version >= 2.1.0 ? ... yes (2.3.3) Your git bin path is "/opt/gitlab/embedded/bin/git" Git version >= 2.7.3 ? ... yes (2.8.4) Active users: 21

Checking GitLab ... Finished

Results of GitLab environment info

System information System: Ubuntu 14.04 Current User: git Using RVM: no Ruby Version: 2.3.3p222 Gem Version: 2.6.6 Bundler Version:1.13.7 Rake Version: 10.5.0 Sidekiq Version:4.2.7

GitLab information Version: 8.15.4 Revision: a0b13790 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: postgresql URL: http://xx.xx.xx.xx HTTP Clone URL: http://xx.xx.xx.xx/some-group/some-project.git SSH Clone URL: git@xx.xx.xx.xx:some-group/some-project.git Using LDAP: no Using Omniauth: no

GitLab Shell Version: 4.1.1 Repository storage paths:

  • default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks/ Git: /opt/gitlab/embedded/bin/git

Possible fixes

Tried reverting back to 8.15.4 and gitlab-ctl revert-pg-upgrade giving 500 error on Project page. All other services works fine.