Backend: Configure Vault with GDK

Context

Since grouppipeline authoring has inherited Category:Secrets Management, it has become clear that there needs to be an easier way to locally test changes to the secrets keyword.

Proposal

This work will need to be done in several segments:

1. Fix GDK's runner configuration when using HTTPS and Docker

Currently, GDK configures a Docker runner incorrectly when using HTTPS. Two changes need to be made:

1. The SSL certificate needs to be mounted to the Docker container when the runner is started
2. tls-ca-file should not be set in gitlab-runner-config.toml when using a Docker runner

2. Add a Vault service to GDK.

This service will be off by default. It can be enabled with:

# gdk.yml

vault:
  enabled: true
  listen_address: 127.0.0.1 # optional, defaults to the top level `listen_address`

gdk start vault will start a Vault dev server with the given listen address

Ideally, this work should also:

• Export a VAULT_ADDR variable so developers can configure Vault on the command line (only when Vault is enabled)
• Ensure that Gitlab.settings.ci_jwt_signing_key is never nil in local environments
• Install Vault with Homebrew (only when Vault is enabled)

3. Add helpers to configure Vault.

The Vault server needs to be configured with an auth policy and role and should have some seed data. The policy and seed data can be configured when the Vault server is started, but the role is project specific. Examples of the configuration can be seen in https://gitlab.com/-/snippets/2380296

1. Seeding data

   vault secrets enable -path=kv-v2 kv-v2
   
   vault kv put kv-v2/gitlab-test/db password=db-password-goes-here
   vault kv put kv-v2/gitlab-test/db other_secret=more-secret-value

2. Seeding auth policy

   vault auth enable -path=gitlab jwt
   
   vault write auth/gitlab/config \
     jwks_url="https://`gdk.hostname`:`gdk.port`/-/jwks" \
     bound_issuer="https://`gdk.hostname`:`gdk.port`"

3. Seeding auth role

   vault write auth/gitlab/role/gitlab-test-role - <<EOF
   {
     "role_type": "jwt",
     "policies": ["gitlab-test-policy"],
     "token_explicit_max_ttl": 600,
     "user_claim": "user_email",
     "bound_claims": {
       "project_id": "<project_id>"
      },
      "bound_audiences": "https://`gdk.hostname`:`gdk.port`"
    }
    EOF

   This either needs to be run manually, or we could add a rake helper, or we can find a way to configure it for every project in the GDK

Implementation Table

Title	MR Link	Done?
Fix GDK's runner configuration when using HTTPS and Docker	gitlab-development-kit!2714 (merged)	✅
Add a Vault service to GDK	gitlab-development-kit!2735 (merged)	✅
Add helpers to configure Vault	gitlab-development-kit!2766 (merged)	✅