Backend: Configure Vault with GDK
Context
Since grouppipeline authoring has inherited Category:Secrets Management, it has become clear that there needs to be an easier way to locally test changes to the secrets
keyword.
Proposal
This work will need to be done in several segments:
1. Fix GDK's runner configuration when using HTTPS and Docker
Currently, GDK configures a Docker runner incorrectly when using HTTPS. Two changes need to be made:
- The SSL certificate needs to be mounted to the Docker container when the runner is started
-
tls-ca-file
should not be set ingitlab-runner-config.toml
when using a Docker runner
2. Add a Vault service to GDK.
This service will be off by default. It can be enabled with:
# gdk.yml
vault:
enabled: true
listen_address: 127.0.0.1 # optional, defaults to the top level `listen_address`
gdk start vault
will start a Vault dev server with the given listen address
Ideally, this work should also:
- Export a
VAULT_ADDR
variable so developers can configure Vault on the command line (only when Vault is enabled) - Ensure that
Gitlab.settings.ci_jwt_signing_key
is nevernil
in local environments - Install Vault with Homebrew (only when Vault is enabled)
3. Add helpers to configure Vault.
The Vault server needs to be configured with an auth policy and role and should have some seed data. The policy and seed data can be configured when the Vault server is started, but the role is project specific. Examples of the configuration can be seen in https://gitlab.com/-/snippets/2380296
-
Seeding data
vault secrets enable -path=kv-v2 kv-v2 vault kv put kv-v2/gitlab-test/db password=db-password-goes-here vault kv put kv-v2/gitlab-test/db other_secret=more-secret-value
-
Seeding auth policy
vault auth enable -path=gitlab jwt vault write auth/gitlab/config \ jwks_url="https://`gdk.hostname`:`gdk.port`/-/jwks" \ bound_issuer="https://`gdk.hostname`:`gdk.port`"
-
Seeding auth role
vault write auth/gitlab/role/gitlab-test-role - <<EOF { "role_type": "jwt", "policies": ["gitlab-test-policy"], "token_explicit_max_ttl": 600, "user_claim": "user_email", "bound_claims": { "project_id": "<project_id>" }, "bound_audiences": "https://`gdk.hostname`:`gdk.port`" } EOF
This either needs to be run manually, or we could add a
rake
helper, or we can find a way to configure it for every project in the GDK
Implementation Table
Title | MR Link | Done? |
---|---|---|
Fix GDK's runner configuration when using HTTPS and Docker | gitlab-development-kit!2714 (merged) | |
Add a Vault service to GDK | gitlab-development-kit!2735 (merged) | |
Add helpers to configure Vault | gitlab-development-kit!2766 (merged) |