Consider the removal of the nginx ingress for the GKE Container Registry
With the recent implementation of the Container Registry into GKE, we went the quick route and utilized the nginx ingress provided by our helm chart in order to quickly utilize this as a PoC. This adds a few things:
- Complexity with the way our haproxy must transfer data to and from the GKE cluster
- Adds an extra network hop that we don't REALLY need:
- Current: GLB -> haproxy -> registry nodes
- Proposed: GLB -> haproxy -> nginx ingress -> pods
- Ningx is not doing anything out of the ordinary other than forwarding traffic
- With this configuration we are using Let's Encrypt on this new external endpoint
registry.gke.<ENV>.gitlab.<ROOT>
. This adds a layer of configuration that we could potentially eliminate - This makes it hard to slowly roll traffic into either our VM's or GKE. Instead our solution is an on/off switch
Proposal
- Configure the container registry without an ingress
- Expose the container registry service with an internal static IP that we can feed to haproxy