Draft: Add id_token sub-keyword to secrets

What does this MR do and why?

This is the final backend MR for #356986 (closed).

Adds id_token keyword (FOSS + EE)

This MR adds an id_token keyword that can be used to define JWTs for a CI job. The syntax looks like this:

job_with_id_tokens:
  secrets:
    FIRST_ID_TOKEN:
      id_token:
        aud: 'https://gitlab.com'
    SECOND_ID_TOKEN:
      id_token:
        aud:
          - 'https://test.com'
          - 'https://development.com'
  script:
    - echo $FIRST_ID_TOKEN
    - echo $SECOND_ID_TOKEN

These ID tokens can be used to authenticate with 3rd parties - the aud sub-keyword can be used to ensure that the token is only useable with the audience for which it is intended. aud can be defined either as a string (single audience token) or an array of strings (multiple audience token).

Adds token keyword (EE only)

The token keyword can be used to specify which ID token should be used to fetch a secret from Hashicorp Vault. The syntax looks like this:

job_with_id_tokens:
  secrets:
    FIRST_ID_TOKEN:
      id_token:
        aud: 'https://production.com'
    SECOND_ID_TOKEN:
      id_token:
        aud:
          - 'https://staging.com'
          - 'https://development.com'
    PROD_DATABASE_PASSWORD:
      vault: prod/db/password
      token: FIRST_ID_TOKEN
    DEV_DATABASE_PASSWORD:
      vault: dev/db/password
      token: SECOND_ID_TOKEN
  script:
    - cat $PROD_DATABASE_PASSWORD
    - cat $DEV_DATABASE_PASSWORD

Screenshots or screen recordings

This configuration will fetch the test secret created when following the GDK Vault docs:

When that job runs, the output will look like this:

How to set up and validate locally

1. Prepare GDK

1. Set Feature.enable(:ci_job_jwt)
2. Make sure Gitlab::CurrentSettings.ci_jwt_signing_key is a private key (it should not be nil)

2. Use HTTPS

The ID token JWTs use JWT V2, which requires HTTPS. You can configure your GDK to use HTTPS by following these instructions: https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/39f962013c8095565f36f3963e50153df44143ed/doc/howto/nginx.md#update-gdkyml-for-https-optional

3. Set up Hashicorp Vault

The easiest way set up and configure Vault is to use the vault:configure Rake task, which has not yet been merged: gitlab-development-kit!2766 (merged)

Follow the instructions in the GDK Vault docs from that MR.

4. Test this MR

You will need to use the same project that you configured with Vault to test this MR.

Update the project's .gitlab-ci.yml file to use the new keywords:

test_secrets:
  variables:
    VAULT_AUTH_PATH: gitlab
    VAULT_AUTH_ROLE: gitlab-test-role
    VAULT_SERVER_URL: <vault_server_url>
  secrets:
    FIRST_ID_TOKEN:
      id_token:
        aud: 'https://aws.com'
    SECOND_ID_TOKEN:
      id_token:
        aud:
          - '<gdk_url>'
          - 'https://gitlab.com'
    DATABASE_PASSWORD:
      vault: gitlab-test/db/password
      token: SECOND_ID_TOKEN  
  script:
    - echo $DATABASE_PASSWORD
    - cat $DATABASE_PASSWORD

Run a pipeline and see that the job echos the database password.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.