Skip to content

Add `token` sub-keyword to CI config

Avielle Wolfe requested to merge 356986-aw-add-token-sub-keyword into master

What does this MR do and why?

This MR is the 5th step for #356986 (closed).

Title MR Link Done?
Spike !94403 (closed)
Add id_tokens field to ci_builds_metadata !94878 (merged)
Pass first ID token to the runner !95254 (merged)
Add id_tokens keyword to .gitlab-ci.yml !103391 (merged)
Add token keyword to .gitlab-ci.yml !103801 (merged) 👋 YOU ARE HERE
Document new keywords

This MR adds a new token sub-keyword to secrets entries that can be used to specify which ID token should be used to fetch a secret from Hashicorp Vault. The syntax looks like this:

job_with_id_tokens:
  id_tokens:
    FIRST_ID_TOKEN:
      aud: 'https://gitlab.com'
    SECOND_ID_TOKEN:
      aud:
        - 'https://test.com'
        - 'https://development.com'
  secrets:
    PROD_DATABASE_PASSWORD:
      vault: prod/db/password
      token: $FIRST_ID_TOKEN  # use the value from FIRST_ID_TOKEN to fetch PROD_DATABASE_PASSWORD
    DEV_DATABASE_PASSWORD:
      vault: dev/db/password
      token: $SECOND_ID_TOKEN  # use the value from SECOND_ID_TOKEN to fetch DEV_DATABASE_PASSWORD
  script:
    - cat $PROD_DATABASE_PASSWORD
    - cat $DEV_DATABASE_PASSWORD

Screenshots or screen recordings

This configuration will fetch the test secret created when following the GDK Vault docs:

Screen_Shot_2022-11-28_at_11.23.13

When that job runs, the output will look like this:

Screenshot_2022-11-11_at_16.05.47

How to set up and validate locally

1. Prepare GDK

  1. Set Feature.enable(:ci_job_jwt)
  2. Make sure Gitlab::CurrentSettings.ci_jwt_signing_key is a private key (it should not be nil)

2. Use HTTPS

The ID token JWTs use JWT V2, which requires HTTPS. You can configure your GDK to use HTTPS by following these instructions: https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/39f962013c8095565f36f3963e50153df44143ed/doc/howto/nginx.md#update-gdkyml-for-https-optional

3. Set up Hashicorp Vault

The easiest way set up and configure Vault is to use the vault:configure Rake task. Follow the instructions in the GDK Vault docs.

4. Test this MR

You will need to use the same project that you configured with Vault to test this MR.

Update the project's .gitlab-ci.yml file to use the new keywords:

test_secrets:
  variables:
    VAULT_AUTH_PATH: gitlab
    VAULT_AUTH_ROLE: gitlab-test-role
    VAULT_SERVER_URL: <vault_server_url>
  id_tokens:
    FIRST_ID_TOKEN:
      aud: https://aws.com
    SECOND_ID_TOKEN:
      aud:
        - https://gitlab.com
        - <gdk_url>
  secrets:
    DATABASE_PASSWORD:
      vault: gitlab-test/db/password
      token: $SECOND_ID_TOKEN
  script:
    - echo $DATABASE_PASSWORD
    - cat $DATABASE_PASSWORD

Run a pipeline and see that the job echos the database password.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Avielle Wolfe

Merge request reports