Additional "External nginx ingress controller" documentation
There's going to be several bits of information in this ticket, but only some of it is actionable. The rest is just informational. I'll start with the actionable.
1. Spec file vs helm chart
The current documentation discusses the ConfigMap necessary to add the TCP mapping for nginx-ingress using a normal k8s spec file. You might consider adding a section for doing the same using the nginx-ingress helm chart as that procedure is poorly described in the nginx-ingress chart docs but is actually easy to do once you understand it:
--set tcp.22="gitlab/mygitlab-gitlab-shell:22"
or in a values.yaml
:
tcp:
22: "gitlab/mygitlab-gitlab-shell:22"
I know this seems obvious once it's written out like that, but for at least one Helm neophyte (i.e. me), this information was hard to track down.
2. Additional entries in Gitlab helm values
The docs mention the global.ingress.class=myingressclass
entry, but there are a few more values that make the process a little more complete.
If using an external ingress controller, then you probably don't want to run the built in one so:
--set nginx-ingress.enabled=false
As an aside, and I know this isn't your chart so you don't control it, but to add to the confusion this is nginx-ingress.enabled=false
whereas for postgresql
, certmanager
, and gitlab-runner
it's install=false
. Then for registry
, we're back to enabled=false
to turn it off. :) Again, I know the two oddballs aren't your charts and install
does seem to make logical sense, but you might consider switching your own charts to enabled
if that turns out to be common standard.
If someone is using an external ingress, then there's at least a decent chance they are also using an external cert-manager instance or some other approach to handling TLS certs, so you might consider mentioning the two following entries:
--set certmanager.install=false
--set global.ingress.configureCertmanager=false
That second one is pretty important and cost me a half day's work figuring out. One might assume that not installing certmanager would disable the built-in certmanager configuration throughout the Gitlab components, however that isn't the case1. Without setting configureCertmanager
to false, the unicorn ingress, at least, will still have annotations that point to nonexistent certificates.
Now, as for the reason why we've had to move to using an external ingress... It wasn't just convenience. In fact, the "convenient" thing to do was to leave thing as close to default as possible, however, for some reason the default configuration was giving us frequent and random TLS handshaking issues. We'd randomly get timeouts trying to connect to the primary Gitlab application from both the runners and any browser. It was maddening. We'd be using the application and then the next click would time out; always on the TLS handshake. It made the application genuinely unusable. We haven't created an issue for this as we were not able to gather ANY useful information as to the cause and the reason could very easily be specific to our environment. I don't want to waste the Gitlab team's time on an issue that I can't provide any useful information for. In any case, after having switched to our own nginx-ingress controller, which uses the same cert-manager and therefore the same TLS cert as the Gitlab setup was using, the problem has not resurfaced. I have no explanation.
In case anyone else runs across this problem, here is all I can think of that might be relevant:
- We're using AWS EKS, so the default gitlab ingress creates an ELB that uses TCP (not HTTP/S) on the listeners
- Our non-gitlab nginx-ingress controller is setup to use an NLB as the entrypoint, so that's possibly related? Can't see how it would be, though.
- Our cert-manager is installed from official helm chart v0.4.1 and is used for both Gitlab ingress controller as well as our non-gitlab controller
- The same Let's Encrypt generated certificate for both gitlab and non-gitlab ingress controller
Given the above, the obvious differences are the nginx-ingress controller installed by gitlab (not sure if there's any customization going on there) and the ELB vs NLB as the internet gateway. Those could be total red herrings, though.
-
After having worked with the Gitlab charts over the past few weeks, I understand this as actually one value disabling the "external" chart, and the other valuing changing how the Gitlab charts behave, but for many, including myself until recently, the thought process of theses values is that we're configuring the "Gitlab" ecosystem as a whole, so the idea that some settings need to be, essentially, duplicated in several places is a bit unexpected; however unavoidable that may be.
↩