geo.staging.gitlab.com 502 on sign in: upstream sent too big header

Problem

• Open a new private window
• Visit https://geo.staging.gitlab.com/dashboard/projects
• Get redirected to https://geo.staging.gitlab.com/users/sign_in
• Submit credentials
• Land on 2nd factor form
• Submit 2nd factor
• The POST to https://geo.staging.gitlab.com/users/sign_in gets a 502

I found this in /var/log/gitlab/nginx/gitlab_error.log (I modified my IP address.):

2022/01/26 00:02:13 [error] 26641#0: *829148 upstream sent too big header while reading response header from upstream, client: 103.111.128.196, server: geo.staging.gitlab.com, request: "POST /users/sign_in HTTP/2.0", upstream: "http://unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket:/users/sign_in", host: "geo.staging.gitlab.com", referrer: "https://geo.staging.gitlab.com/users/sign_in"

Internal Slack conversation: https://gitlab.slack.com/archives/CQKEEQKNE/p1642552795003900