feat: configurable resources for OpenSearch and NGINX

Description

• Adds configuration options for NGINX and OpenSearch so that operators can update their CPU requests/limits.
• Adds documentation for the above and the existing configuration options to the new "Cluster Configuration" page.
• Fix the CI commitlint process.

Supporting information

• https://tasks.opencraft.com/browse/BB-7418
• tutor-contrib-grove!33 (merged)

Testing instructions

The changes have been deployed to the cluster https://gitlab.com/opencraft/devstacks/keith/grove-do.

• Clone the repo. Secrets are in vault.
• Verify that caddy pods are scheduled on the same node as the nginx-controller pod:

  ./kubectl get pods -A -o wide | grep -P "caddy|nginx-controller|NODE"
  NAMESPACE                 NAME                                                          READY   STATUS    RESTARTS      AGE   IP             NODE                      NOMINATED NODE   READINESS GATES
  es1                       caddy-f747bf555-5rmcm                                         1/1     Running   0               16h     10.244.1.37    kgcluster-workers-ff9n4   <none>           <none>
  es2                       caddy-5b67874685-flvbg                                        1/1     Running   0               59m     10.244.1.45    kgcluster-workers-ff9n4   <none>           <none>
  kube-system               ingress-nginx-controller-7f55686c4f-jxpnb                     1/1     Running   0               17h     10.244.1.5     kgcluster-workers-ff9n4   <none>           <none>

• Verify that the redis pods have been scheduled to nodes where neither OpenSearch or ElasticSearch are running.

  kubectl get pods -A -o wide | grep -P "redis|opensearch-cluster-master|elasticsearch|NODE"
  NAMESPACE                 NAME                                                          READY   STATUS    RESTARTS      AGE   IP             NODE                      NOMINATED NODE   READINESS GATES
  es1                       elasticsearch-68cdbcc775-xxgrc                                1/1     Running   0               16h     10.244.0.91    kgcluster-workers-ff9nv   <none>           <none>
  es1                       redis-f9b7fdffc-zcl92                                         1/1     Running   0               16h     10.244.1.85    kgcluster-workers-ff9n4   <none>           <none>
  es2                       elasticsearch-76d496b678-5rvz6                                1/1     Running   0               16h     10.244.0.193   kgcluster-workers-ff9ni   <none>           <none>
  es2                       redis-7896dcc5dd-g5xnk                                        1/1     Running   0               16h     10.244.1.71    kgcluster-workers-ff9n4   <none>           <none>
  monitoring                opensearch-cluster-master-0                                   1/1     Running   0               3d22h   10.244.0.197   kgcluster-workers-ff9ni   <none>           <none>

• Check that the HPA has been created for caddy on the es2 namespace.

  kubectl describe hpa -nes2 caddy-hpa

• Check that you can change the resource quotas in private.yml to any valid value and run ./tf plan && ./tf apply.
• Run mkdocs serve and proofread the new docs at http://localhost:8000/user-guides/cluster-configuration/

Checklist

If any of the items below is not applicable, do not remove them, but put a check in it.

• All providers include the new feature/change
• All affected providers can provision new clusters
• Unit tests are added/updated
• Documentation is added/updated
• The TOOLS_CONTAINER_IMAGE_VERSION in ci_vars.yml is updated
• The grove-template repository is updated