Advertise grpcs:// to agents by default and render the KAS gRPC Ingress for NGINX

Vishal Takrequested to merge
vtak/kas-grpcs-default into master

What does this MR do?

Makes native gRPC the default agent connection for chart installations, part of Phase 4 of gitlab-org&23445 (closed) (gitlab-org/gitlab#628216 (closed)).

  • gitlab_kas.external_url defaults to grpcs://<kas host> when the chart routes gRPC to KAS: with Gateway API (default), or with the NGINX Ingress provider. Other Ingress controllers and relative URL roots keep wss://; plain HTTP keeps ws://. global.appConfig.gitlab_kas.externalUrl still overrides everything.
  • The KAS gRPC Ingress is rendered by default for the NGINX provider. The toggle gains a global form, global.kas.ingress.grpc.enabled (unset = NGINX only, true, false), because the URL is derived from the webservice, sidekiq and toolbox charts, which cannot see the kas chart's values. gitlab.kas.ingress.grpc.enabled keeps precedence for the Ingress itself.
  • Docs: agent connection protocol section, gRPC Ingress section, globals, and a 10.4.0 upgrade note.

Agents already connected over wss:// are unaffected; KAS keeps serving both protocols on one port.

Validation

kas-grpc-chart.sh

Logs

$  PIN_KAS_HOST_TO_NODE=1 ./kas-grpc-chart.sh run
docker resources: 16 CPUs, 15 GB memory
helm v4.0.5+g1b6053d | Client Version: v1.35.0 | k3d version v5.8.3

== 0. Inputs ==
chart: /Users/vtak/dev/gitlab-org/charts/gitlab @ 19e412cfa (vtak/kas-grpcs-default)
domain: 192.168.0.150.nip.io | GitLab: https://gitlab-kas.192.168.0.150.nip.io | KAS: kas-kas.192.168.0.150.nip.io | agentk: gitlab/gitlab-agent v19.3.1
branch has the KAS ClientTrafficPolicy

== 1. k3d cluster (host port 443 -> Envoy / NGINX) ==
OK   cluster ready

== 2. Self-signed wildcard certificate as every listener's secret (this is what triggers GEP-3567) ==
OK   secrets created

== 3. External components with the chart's CI scripts (CloudNativePG, Valkey, Garage) ==
(log: /Users/vtak/Desktop/kas-grpc-check/kas-grpc-chart-check/external-components.log)
OK   PostgreSQL, Valkey and Garage ready

== 4. Deploy GitLab, Gateway API with the bundled Envoy Gateway (15-20 min, pulls the CNG images) ==
   2 Completed   16 Running 
-- advertised KAS settings:
external_url: "grpcs://kas-kas.192.168.0.150.nip.io"
internal_url: "grpc://dev-gitlab-kas.default.svc:8153"
OK   gitlab.yml advertises grpcs://kas-kas.192.168.0.150.nip.io
-- Gateway listener condition: OverlappingCertificates
-- KAS policies:
BackendTrafficPolicy   dev-gitlab-kas              dev-gitlab-kas      <none>       <none>          true
ClientTrafficPolicy    dev-gitlab-kas-ctp          dev-gitlab-gw       kas-web      [h2 http/1.1]   <none>
-- ALPN: KAS host ALPN protocol: h2 | GitLab host ALPN protocol: http/1.1
OK   KAS listener negotiates HTTP/2
OK   Gateway API: HTTP/2 gRPC answered by KAS (grpc-message: Request unauthenticated with bearer)
OK   Gateway API: WebSocket upgrade 101

== 5. Real agents over grpcs:// and wss:// through Envoy ==
OK   agent 1 registered in root/kas-grpc-check
PIN_KAS_HOST_TO_NODE is set: agent pods resolve kas-kas.192.168.0.150.nip.io to the cluster node 172.18.0.3 (test scaffolding, not the real DNS path)
agentk-grpc -> grpcs://kas-kas.192.168.0.150.nip.io (TLS verified against the test CA)
agentk-ws -> wss://kas-kas.192.168.0.150.nip.io (TLS verified against the test CA)
connected agentk instances: 2
  pod=agentk-grpc-gitlab-agent-v2-97cdb48b-7llwf version=v19.3.1
  pod=agentk-ws-gitlab-agent-v2-77bbf89496-x58r4 version=v19.3.1
OK   Gateway API: agentk-grpc connected over grpcs:// (warnings: 0)
OK   Gateway API: agentk-ws connected over wss:// (warnings: 0)

== 6. NGINX Ingress instead of Gateway API (gRPC Ingress rendered by default) ==
-- advertised KAS settings:
external_url: "grpcs://kas-kas.192.168.0.150.nip.io"
internal_url: "grpc://dev-gitlab-kas.default.svc:8153"
OK   gitlab.yml still advertises grpcs://kas-kas.192.168.0.150.nip.io
NAME                  PATH                   BACKEND
dev-gitlab-kas-grpc   /gitlab\.agent\.(.+)   GRPC
OK   gRPC Ingress rendered
-- ALPN: KAS host ALPN protocol: h2
OK   NGINX Ingress: HTTP/2 gRPC answered by KAS (grpc-message: Request unauthenticated with bearer)
OK   NGINX Ingress: WebSocket upgrade 101
connected agentk instances: 3
  pod=agentk-grpc-gitlab-agent-v2-97cdb48b-7llwf version=v19.3.1
  pod=agentk-grpc-gitlab-agent-v2-97cdb48b-k2rrp version=v19.3.1
  pod=agentk-ws-gitlab-agent-v2-77bbf89496-f8z8d version=v19.3.1
OK   NGINX Ingress: agentk-grpc connected over grpcs:// (warnings: 0)
OK   NGINX Ingress: agentk-ws connected over wss:// (warnings: 1)

== 7. Opt out: global.kas.ingress.grpc.enabled=false falls back to wss:// ==
external_url: "wss://kas-kas.192.168.0.150.nip.io"
internal_url: "grpc://dev-gitlab-kas.default.svc:8153"
OK   gitlab.yml advertises wss://kas-kas.192.168.0.150.nip.io with the toggle off
OK   gRPC Ingress not rendered

== Done. Everything passed. Logs in /Users/vtak/Desktop/kas-grpc-check/kas-grpc-chart-check. Run './kas-grpc-chart.sh cleanup' to delete the cluster. ==
$

Related issues

Related to gitlab-org/gitlab#628216 (closed)

Author checklist

  • Merge Request Title and Description are up to date, accurate, and descriptive.
  • MR targeting the appropriate branch.
  • MR has a green pipeline.
  • Documentation created/updated.
  • Tests added/updated.
  • Validated end-to-end on a real cluster deployment. Not done; see Validation.
Edited by Nailia Iskhakova

Merge request reports