Skip to content

[CI] Ensure NGINX instances do not collide

Mitchell Nielsen requested to merge 3047-ci-fix-ingress into master

What does this MR do?

Ensures support for multiple instance of NGINX Ingress Controller within the same namespace for CI.

We set:

nginx-ingress:
  controller:
    electionID: $RELEASE_NAME
    ingressClassByName: true
    ingressClassResource:
      controllerValue: ci.gitlab.com/$RELEASE_NAME

These ensure that leader election is separate between instances, and ensure that the ingress class is unique per instance.

This follows the recommendation from the NGINX documentation.

Changelog: fixed

Related issues

Closes #3047 (closed)

Testing

The following tests are in the context of CI's GKE cluster, where gke-production release is running (to validate that changes work when another instance of NGINX is running in the same namespace).

Pipeline

Full pipeline run: https://gitlab.com/gitlab-org/charts/gitlab/-/pipelines/433496286

Ingresses

Confirming Ingresses have Addresses assigned:

$ kubectl get svc -n helm-charts-win -lrelease=rvw-3047-ci-fix-ingress,app=nginx-ingress,component=controller
NAME                                                       TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)                                   AGE
rvw-3047-ci-fix-ingress-nginx-ingress-controller           LoadBalancer   10.43.255.60   34.142.108.129   80:31366/TCP,443:30467/TCP,22:31832/TCP   23m
rvw-3047-ci-fix-ingress-nginx-ingress-controller-metrics   ClusterIP      10.43.247.39   <none>           10254/TCP                                 23m

$ # expected external IP is 34.142.108.129

$ kubectl get ingress -n helm-charts-win -lrelease=rvw-3047-ci-fix-ingress
NAME                                         CLASS                           HOSTS                                               ADDRESS          PORTS     AGE
rvw-3047-ci-fix-ingress-minio                rvw-3047-ci-fix-ingress-nginx   minio-gke-review-3047-c-p93rp7.helm-charts.win      34.142.108.129   80, 443   22m
rvw-3047-ci-fix-ingress-registry             rvw-3047-ci-fix-ingress-nginx   registry-gke-review-3047-c-p93rp7.helm-charts.win   34.142.108.129   80, 443   22m
rvw-3047-ci-fix-ingress-webservice-default   rvw-3047-ci-fix-ingress-nginx   gitlab-gke-review-3047-c-p93rp7.helm-charts.win     34.142.108.129   80, 443   22m

Endpoint response

$ curl -Iv https://gitlab-gke-review-3047-c-p93rp7.helm-charts.win
$ curl -Iv https://gitlab-gke-review-3047-c-p93rp7.helm-charts.win
*   Trying 34.142.108.129:443...
* Connected to gitlab-gke-review-3047-c-p93rp7.helm-charts.win (34.142.108.129) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=helm-charts.win
*  start date: Nov 23 23:53:01 2021 GMT
*  expire date: Feb 21 23:53:00 2022 GMT
*  subjectAltName: host "gitlab-gke-review-3047-c-p93rp7.helm-charts.win" matched cert's "*.helm-charts.win"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7ff5f8811600)
> HEAD / HTTP/2
> Host: gitlab-gke-review-3047-c-p93rp7.helm-charts.win
> user-agent: curl/7.77.0
> accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 302
HTTP/2 302
< date: Mon, 20 Dec 2021 17:47:37 GMT
date: Mon, 20 Dec 2021 17:47:37 GMT
< content-type: text/html; charset=utf-8
content-type: text/html; charset=utf-8
< location: https://gitlab-gke-review-3047-c-p93rp7.helm-charts.win/users/sign_in
location: https://gitlab-gke-review-3047-c-p93rp7.helm-charts.win/users/sign_in
< cache-control: no-cache
cache-control: no-cache
< pragma: no-cache
pragma: no-cache
< referrer-policy: strict-origin-when-cross-origin
referrer-policy: strict-origin-when-cross-origin
< x-content-type-options: nosniff
x-content-type-options: nosniff
< x-download-options: noopen
x-download-options: noopen
< x-frame-options: DENY
x-frame-options: DENY
< x-permitted-cross-domain-policies: none
x-permitted-cross-domain-policies: none
< x-request-id: 01FQCF9EVWRFBDN6P804MVYG2E
x-request-id: 01FQCF9EVWRFBDN6P804MVYG2E
< x-runtime: 0.017248
x-runtime: 0.017248
< x-ua-compatible: IE=edge
x-ua-compatible: IE=edge
< x-xss-protection: 1; mode=block
x-xss-protection: 1; mode=block
< strict-transport-security: max-age=63072000
strict-transport-security: max-age=63072000

<
* Connection #0 to host gitlab-gke-review-3047-c-p93rp7.helm-charts.win left intact

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Required

  • Merge Request Title and Description are up to date, accurate, and descriptive
  • MR targeting the appropriate branch
  • MR has a green pipeline on GitLab.com

Expected (please provide an explanation if not completing)

  • Test plan indicating conditions for success has been posted and passes
  • Documentation created/updated
  • Tests added
  • Integration tests added to GitLab QA
  • Equivalent MR/issue for omnibus-gitlab opened

Closes #3047 (closed)

Edited by Mitchell Nielsen

Merge request reports