Kubernetes integration and Ingress deployment with Ingress Endpoint showing a spinner forever
What happened?
I have a Kubernetes cluster built using kubeadm on VPS with a public IP address for each.
I'm configuring Gitlab in order to manage it in order to run the tests and auto deploy my apps. The integration configuration at group level went well, and I was able to deploy cert-manager and the gitlab runner and the build run.
Now I'm trying to make working the ingress part, so I've clicked the Ingress "Install" button from the "Applications" tab and it has first failed as it couldn't get the ingress-nginx-ingress-controller Service working.
What I did is then trigger again the installation and edit the service in order to add the spec.externalIPs attribute with the public IP addresses of my worker nodes and the installation finished on success.
But now when I come back to the "Applications" tab of the Kubernetes cluster integration, I see an "Ingress Endpoint" field with a spinner, remaining forever, like the following:
Based on the screenshot from the Web Application Firewall (ModSecurity) documentation I should get the load balancer external IP address I guess, which would be the IP addresses from my worker nodes in my case.
Created things in my cluster so far
NAME READY STATUS RESTARTS AGE
pod/certmanager-cainjector-8c559d68f-fq6cm 1/1 Running 297 6d
pod/certmanager-cert-manager-855454cc95-422s2 1/1 Running 5 6d
pod/certmanager-cert-manager-webhook-84545b7b88-glt99 1/1 Running 1 6d
pod/ingress-nginx-ingress-controller-75686bdf76-9cwzd 2/2 Running 0 149m
pod/ingress-nginx-ingress-default-backend-c9b59c85-lfkpx 1/1 Running 0 149m
pod/runner-gitlab-runner-7b9b4d5dbd-ln9rn 1/1 Running 0 5d9h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/certmanager-cert-manager ClusterIP 10.110.46.114 <none> 9402/TCP 6d
service/certmanager-cert-manager-webhook ClusterIP 10.98.198.68 <none> 443/TCP 6d
service/ingress-nginx-ingress-controller LoadBalancer 10.111.213.120 <worker 1 public IP address>,<worker 2 public IP address>,<worker 3 public IP address> 80:31580/TCP,443:31765/TCP 149m
service/ingress-nginx-ingress-default-backend ClusterIP 10.104.157.122 <none> 80/TCP 149m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/certmanager-cainjector 1/1 1 1 6d
deployment.apps/certmanager-cert-manager 1/1 1 1 6d
deployment.apps/certmanager-cert-manager-webhook 1/1 1 1 6d
deployment.apps/ingress-nginx-ingress-controller 1/1 1 1 149m
deployment.apps/ingress-nginx-ingress-default-backend 1/1 1 1 149m
deployment.apps/runner-gitlab-runner 1/1 1 1 6d
NAME DESIRED CURRENT READY AGE
replicaset.apps/certmanager-cainjector-8c559d68f 1 1 1 6d
replicaset.apps/certmanager-cert-manager-855454cc95 1 1 1 6d
replicaset.apps/certmanager-cert-manager-webhook-84545b7b88 1 1 1 6d
replicaset.apps/ingress-nginx-ingress-controller-75686bdf76 1 1 1 149m
replicaset.apps/ingress-nginx-ingress-default-backend-c9b59c85 1 1 1 149m
replicaset.apps/runner-gitlab-runner-7b9b4d5dbd 1 1 1 6dpod/ingress-nginx-ingress-controller-75686bdf76-9cwzd logs:
$ kubectl logs pod/ingress-nginx-ingress-controller-75686bdf76-9cwzd -c nginx-ingress-controller --namespace=gitlab-managed-apps
-------------------------------------------------------------------------------
NGINX Ingress controller
Release: 0.32.0
Build: git-446845114
Repository: https://github.com/kubernetes/ingress-nginx
nginx version: nginx/1.17.10
-------------------------------------------------------------------------------
I0310 12:24:49.710516 6 flags.go:204] Watching for Ingress class: nginx
W0310 12:24:49.711913 6 flags.go:249] SSL certificate chain completion is disabled (--enable-ssl-chain-completion=false)
W0310 12:24:49.713848 6 client_config.go:543] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
I0310 12:24:49.715095 6 main.go:220] Creating API client for https://10.96.0.1:443
I0310 12:24:49.748901 6 main.go:264] Running in Kubernetes cluster version v1.20 (v1.20.4) - git (clean) commit e87da0bd6e03ec3fea7933c4b5263d151aafd07c - platform linux/amd64
I0310 12:24:49.760051 6 main.go:94] Validated gitlab-managed-apps/ingress-nginx-ingress-default-backend as the default backend.
I0310 12:24:49.993311 6 main.go:105] SSL fake certificate created /etc/ingress-controller/ssl/default-fake-certificate.pem
I0310 12:24:50.000806 6 main.go:113] Enabling new Ingress features available since Kubernetes v1.18
E0310 12:24:50.008445 6 main.go:122] Unexpected error searching IngressClass: ingressclasses.networking.k8s.io "nginx" is forbidden: User "system:serviceaccount:gitlab-managed-apps:ingress-nginx-ingress" cannot get resource "ingressclasses" in API group "networking.k8s.io" at the cluster scope
W0310 12:24:50.008501 6 main.go:125] No IngressClass resource with name nginx found. Only annotation will be used.
I0310 12:24:50.111830 6 nginx.go:263] Starting NGINX Ingress controller
I0310 12:24:50.207068 6 event.go:278] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"gitlab-managed-apps", Name:"ingress-nginx-ingress-controller", UID:"dbaee415-85e9-482b-8845-15b712f0f33e", APIVersion:"v1", ResourceVersion:"1457760", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap gitlab-managed-apps/ingress-nginx-ingress-controller
I0310 12:24:51.313126 6 nginx.go:307] Starting NGINX process
I0310 12:24:51.313193 6 leaderelection.go:242] attempting to acquire leader lease gitlab-managed-apps/ingress-controller-leader-nginx...
I0310 12:24:51.329518 6 status.go:86] new leader elected: ingress-nginx-ingress-controller-75686bdf76-kx94c
W0310 12:24:51.335446 6 controller.go:393] Service "gitlab-managed-apps/ingress-nginx-ingress-default-backend" does not have any active Endpoint
I0310 12:24:51.335615 6 controller.go:139] Configuration changes detected, backend reload required.
I0310 12:24:52.334977 6 controller.go:155] Backend successfully reloaded.
I0310 12:24:52.335057 6 controller.go:164] Initial sync, sleeping for 1 second.
I0310 12:25:35.359006 6 leaderelection.go:252] successfully acquired lease gitlab-managed-apps/ingress-controller-leader-nginx
I0310 12:25:35.359050 6 status.go:86] new leader elected: ingress-nginx-ingress-controller-75686bdf76-9cwzd
W0310 14:14:26.047722 6 controller.go:393] Service "gitlab-managed-apps/ingress-nginx-ingress-default-backend" does not have any active Endpoint
W0310 14:14:29.381509 6 controller.go:393] Service "gitlab-managed-apps/ingress-nginx-ingress-default-backend" does not have any active Endpoint
W0310 14:14:36.078928 6 controller.go:393] Service "gitlab-managed-apps/ingress-nginx-ingress-default-backend" does not have any active Endpoint
W0310 14:14:39.412502 6 controller.go:393] Service "gitlab-managed-apps/ingress-nginx-ingress-default-backend" does not have any active Endpointingress-nginx-ingress-controller service:
apiVersion: v1
kind: Service
metadata:
annotations:
meta.helm.sh/release-name: ingress
meta.helm.sh/release-namespace: gitlab-managed-apps
creationTimestamp: "2021-03-10T12:24:47Z"
labels:
app: nginx-ingress
app.kubernetes.io/managed-by: Helm
chart: nginx-ingress-1.40.2
component: controller
heritage: Helm
release: ingress
managedFields:
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.: {}
f:meta.helm.sh/release-name: {}
f:meta.helm.sh/release-namespace: {}
f:labels:
.: {}
f:app: {}
f:app.kubernetes.io/managed-by: {}
f:chart: {}
f:component: {}
f:heritage: {}
f:release: {}
f:spec:
f:externalTrafficPolicy: {}
f:ports:
.: {}
k:{"port":80,"protocol":"TCP"}:
.: {}
f:name: {}
f:port: {}
f:protocol: {}
f:targetPort: {}
k:{"port":443,"protocol":"TCP"}:
.: {}
f:name: {}
f:port: {}
f:protocol: {}
f:targetPort: {}
f:selector:
.: {}
f:app: {}
f:app.kubernetes.io/component: {}
f:release: {}
f:sessionAffinity: {}
f:type: {}
manager: Go-http-client
operation: Update
time: "2021-03-10T12:24:47Z"
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:spec:
f:externalIPs: {}
manager: kubectl-edit
operation: Update
time: "2021-03-10T12:26:25Z"
name: ingress-nginx-ingress-controller
namespace: gitlab-managed-apps
resourceVersion: "1458172"
uid: 8df23520-2200-4189-bbf6-7d4a9c7b1434
spec:
clusterIP: 10.111.213.120
clusterIPs:
- 10.111.213.120
externalIPs:
- worker 1 public IP address
- worker 2 public IP address
- worker 3 public IP address
externalTrafficPolicy: Cluster
ports:
- name: http
nodePort: 31580
port: 80
protocol: TCP
targetPort: http
- name: https
nodePort: 31765
port: 443
protocol: TCP
targetPort: https
selector:
app: nginx-ingress
app.kubernetes.io/component: controller
release: ingress
sessionAffinity: None
type: LoadBalancer
status:
loadBalancer: {}Like this SO question stated, the issue should come from the fact that status.loadBalancer is empty since the "Install Ingress" section from the "Getting started with Auto DevOps" documentation says that after few minutes the load balancer gets an IP address and it can be retrieved from that service from .status.loadBalancer.ingress[].ip.
So how could I get gitlab.com getting my load balancer IP address(es) in my use case?