Update example values for Google and Amazon Ingress Controllers

What does this MR do?

GKE

Updates the example values file for using the GKE Ingress Controller.

Previously, you needed to ensure that spec.ingressClassName was either set to "" or not set at all.

The custom ingress controller docs now directly reference spec.ingressClassName: https://cloud.google.com/kubernetes-engine/docs/how-to/custom-ingress-controller#ingress_class)

Setting spec.ingressClassName="" is no longer a defined scenario. So to keep things simple, this change removes that example from the example values file. It also expands the notes in the file to more clearly explain the configuration.

Note that "" can still be configured at the moment - that ability has not been removed.

AWS

Updates the AWS Ingress Controller / Load Balancer Controller example values file to reflect that the preferred configuartion approach is to define spec.ingressClassName now that the annotation is deprecated.

Test plan

Ensure the example values files still template successfully:

helm template gitlab . --set certmanager-issuer.email=a@b.com -f examples/aws/ingress.yaml
helm template gitlab . --set certmanager-issuer.email=a@b.com -f examples/gke/ingress.yaml

This should be a sufficient test because spec/configuration/ingress_spec.rb already ensures that the keys affect the rendered objects as expected.

For more thorough testing you can also deploy a simple Ingress object to each cloud provider and confirm that the suggested values do what you expect:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-spec
spec:
  rules:
  - host: cafe.example.com

From this base config, you can mix and match specifying spec.ingressClassName and/or metadata.annotations."kubernetes.io/ingress.class".

In scenarios where you want the cloud provider to pick up the Ingress, you should eventually see an associated IP address as well as some Events on the object.

In the scenario where you want the cloud provider to ignore the Ingress, you should never see an associated IP address appear and should not see any Events on the object (unless you happen to have an Ingress Controller deployed that matches the specified ingress class name).

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Required

• Merge Request Title and Description are up to date, accurate, and descriptive
• MR targeting the appropriate branch
• MR has a green pipeline on GitLab.com
• When ready for review, MR is labeled "~workflow::ready for review" per the Distribution MR workflow

Expected (please provide an explanation if not completing)

• Test plan indicating conditions for success has been posted and passes
• Documentation created/updated
• Tests added
• Integration tests added to GitLab QA
• Equivalent MR/issue for omnibus-gitlab opened
• Validate potential values for new configuration settings. Formats such as integer 10, duration 10s, URI scheme://user:passwd@host:port may require quotation or other special handling when rendered in a template and written to a configuration file.