webservice metrics not responding

Summary

The webservice pod is not listening for metrics requests.

I have found a lot of issues with your metrics configuration (see !3079 (merged) for postgres and !3081 (closed) for gitlab-runner), but this one I can't figure out. You are correctly setting metrics.enabled to true, I can see the port is present on the pod, and the optional section is in the configmap. However the pod does not listen on the metrics port, which I don't know how to diagnose.

Steps to reproduce

Install GitLab with default values. The scrape annotations are there but the gitlab-webservice-default-xxx-yyy pod does not listen on the metrics port, 8083.

Configuration used

global:
  edition: ce
  hosts:
    domain: hsrn.nyu.edu
    hostSuffix: gitlab-test
  ingress:
    class: haproxy
    configureCertmanager: false
    provider: haproxy
    tls:
      enabled: false # haproxy does TLS, no need to do anything here
    annotations:
      haproxy.org/forwarded-for: "true"
      haproxy.org/ssl-redirect: "true"
      haproxy.org/ssl-redirect-code: "301"
      haproxy.org/response-set-header: |
        Strict-Transport-Security "max-age=31536000"
  pages:
    enabled: true
    host: sites-test.hsrn.nyu.edu
    externalHttp:
      - "216.165.12.44"
    externalHttps:
      - "216.165.12.44"

certmanager:
  installCRDs: false
  install: false

nginx-ingress:
  enabled: false

gitlab:
  gitaly:
    persistence:
      size: 50Gi

  webservice:
    trusted_proxies:
      - "10.0.0.0/16"
    workhorse:
      trustedCIDRsForXForwardedFor:
        - "10.0.0.0/16"

  sidekiq:
    trusted_proxies:
      - "10.0.0.0/16"

  gitlab-pages:
    service:
      annotations:
        metallb.universe.tf/address-pool: public
    ingress:
      tls:
        secretName: tls-sites.hsrn.nyu.edu

gitlab-runner:
  metrics:
    enabled: true

postgresql:
  persistence:
    size: 20Gi
  metrics:
    enabled: true
    service:
      annotations:
        gitlab.com/prometheus_scrape: "true"
        gitlab.com/prometheus_port: "9187"

minio:
  persistence:
    size: 100Gi

redis:
  master:
    persistence:
      size: 20Gi

prometheus:
  server:
    persistentVolume:
      size: 10Gi

Versions

• Chart: 6.10.1
• Platform:
  • Self-hosted
• Kubernetes: (kubectl version)
  • Client: 1.25.8
  • Server: 1.25.8
• Helm: (helm version)
  • Client: 3.10.3
  • Server: what's that?