Add support for basic auth GitLab -> Zoekt

What does this MR do?

This is the GitLab side of gitlab-org/cloud-native/charts/gitlab-zoekt!14 (merged) . In summary GitLab consumes Zoekt via a HTTP API. This MR introduces a basic auth username and password authentication mechanism between the two of them. In gitlab-org/cloud-native/charts/gitlab-zoekt!14 (merged) we already configured Zoekt to authenticate based on a mounted secret password if it is enabled. In this MR we enable this setting in Zoekt and we mount the secret automatically in templates/shared-secrets/_generate_secrets.sh.tpl.

Related issues

gitlab-org/gitlab#389749 (closed)

How to test

How to test

1. Update the dependency:

   helm dependency update

2. Create an adequate K8s cluster for testing

   gcloud container clusters create --cluster-version 1.21 --node-version 1.21 --region us-west1-b dylan-zoekt-test-2023-06-29-2 --machine-type n2d-standard-4

3. Install this chart:

   helm install gitlab . --set gitlab-zoekt.install=true --set certmanager-issuer.email="example@example.com"

4. Connect to the toolbox container:

   kubectl exec -it gitlab-toolbox-6d96748dbb-7b2sq bash # Replace with your toolbox pod name

5. Confirm it has the username and password:

   cat /etc/gitlab/zoekt/.gitlab_zoekt_username /etc/gitlab/zoekt/.gitlab_zoekt_password

6. Connect to the zoekt gateway:

   kubectl exec -it gitlab-gitlab-zoekt-0 -c zoekt-gateway bash

7. Confirm it has the htpasswd file:

   cat /etc/nginx/basicauth/.htpasswd

8. From the toolbox, test curling the Zoekt API without credentials

   curl -XPOST -d '{"Q":"gitaly"}' 'http://gitlab-gitlab-zoekt:8080/api/search' -i # Should fail with 401

9. From the toolbox, test curling the Zoekt API with credentials

   curl -XPOST -d '{"Q":"gitaly"}' "http://$(cat /etc/gitlab/zoekt/.gitlab_zoekt_username):$(cat /etc/gitlab/zoekt/.gitlab_zoekt_password)@gitlab-gitlab-zoekt:8080/api/search" -i

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Required

• Merge Request Title and Description are up to date, accurate, and descriptive
• MR targeting the appropriate branch
• MR has a green pipeline on GitLab.com
• When ready for review, MR is labeled "~workflow::ready for review" per the Distribution MR workflow

Expected (please provide an explanation if not completing)

• Test plan indicating conditions for success has been posted and passes
• Documentation created/updated
• Tests added
  • I took a look around for existing tests for similar behaviour. I'm open to adding tests if we have a specific suggestion about how/what to test. I was thinking that helm test might be interesting as it runs in the cluster and could verify that all the secrets exist.
• Integration tests added to GitLab QA
  • Zoekt is not yet covered by integration tests in GitLab QA but once we have test coverage in there for the feature then it should also cover the integration working properly in the charts
• Equivalent MR/issue for omnibus-gitlab opened
  • Zoekt is not supported in Omnibus yet
• Validate potential values for new configuration settings. Formats such as integer 10, duration 10s, URI scheme://user:passwd@host:port may require quotation or other special handling when rendered in a template and written to a configuration file.