Skip to content

Add a prefix to CI Build tokens behind a feature flag

Nick Malcolm requested to merge 426137-prefix-ci-tokens into master

What does this MR do and why?

Add a prefix to CI Build tokens behind a feature flag

Prefixes CI Build tokens (a.k.a. CI_JOB_TOKEN) with glcbt- following the guidance at https://docs.gitlab.com/ee/development/secure_coding_guidelines.html#token-prefixes.

GitLab applies a prefix to some of its generated secrets. For example, a Personal Access Token begins with glpat-. This MR adds a prefix to Build Tokens. It also updates our frontend secret detection which helps prevent users from leaking tokens via Issue / MR comments.

Build tokens belong to build jobs and are used to authenticate against the APIs described at https://docs.gitlab.com/ee/ci/jobs/ci_job_token.html

Build tokens were already prefixed with a hexadecimal partition ID. The new static prefix is placed before the existing prefix.

A feature flag is being used to reduce the risk of breaking CI pipelines and/or third-party integrations, which might have made assumptions about the format of GitLab's build tokens remaining static. The flag can be enabled or disabled per namespace project.

Resolves Add prefix to CI Job Tokens (#426137 - closed). The issue includes discussion of the prefix and risks of change.

Changelog: changed

Rollout issue: [Feature flag] Rollout of `prefix_ci_build_tokens` (gitlab-com/gl-infra/production#17299 - closed)

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

I have followed the checklist. I believe the risks are mitigated by rolling out with a group-based FF.

Screenshots or screen recordings

This change is invisible to the user - CI_JOB_TOKEN should never be seen, only used behind the scenes when jobs are run.

How to set up and validate locally

Note: the tokens in these steps are all A) already expired because their jobs finished and B) for the local GDK development environment. There is no security risk including them, and it assists with understanding the change.

Validating the current state:

  1. Ensure a runner is configured and running locally: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/runner.md
  2. Create a project
  3. Click add file, use the file name .gitlab-ci.yml, and the content below
    # To validate build token prefix.
    # Don't run on .com or you'll leak a sensitive value.
    before_script:
        - apk add --no-cache curl
    insecure:
      stage: build
      script:
        - echo "Attempting to echo CI_JOB_TOKEN."
        - echo $CI_JOB_TOKEN
        - echo "Attempting to echo base64(CI_JOB_TOKEN)"
        - echo $CI_JOB_TOKEN | base64
        - 'curl --location --insecure --output artifacts.zip "https://gdk.test:3443/api/v4/projects/$CI_PROJECT_ID/jobs/artifacts/main/download?job=insecure&job_token=$CI_JOB_TOKEN"'
  4. Commit to main/master
  5. View the job
    $ echo "Attempting to echo CI_JOB_TOKEN."
    Attempting to echo CI_JOB_TOKEN.
    $ echo $CI_JOB_TOKEN
    [MASKED]
    $ echo "Attempting to echo base64(CI_JOB_TOKEN)"
    Attempting to echo base64(CI_JOB_TOKEN)
    $ echo $CI_JOB_TOKEN | base64
    NjRfM1JMM0dBOGFzQmFLNFlvbnp2eWcK
    1. Note that the raw CI_JOB_TOKEN is masked in output (great!)
    2. Note that we can avoid the masking by base64 encoding it first (insecure!)
    3. Note that you can decode the base64 version and get the plaintext token (expected)
      % echo "NjRfM1JMM0dBOGFzQmFLNFlvbnp2eWcK" | base64 -d
      64_3RL3GA8asBaK4Yonzvyg
    4. Note that cURL executes successfully; CI_JOB_TOKEN is valid authentication/authorization

Validating the change:

  1. Check out 426137-prefix-ci-tokens and gdk restart.
  2. Enable the feature flag for a group that isn't a parent of the project, e.g. Feature.enable(:prefix_ci_build_tokens, Group.first)
    1. Re-run the job
    2. Observe that the job token echoed out still lacks a prefix
      ...
      $ echo $CI_JOB_TOKEN | base64
      NjRfb29lbnd4b01MZFRBcmtfR3Jqak4K
      % echo "NjRfb29lbnd4b01MZFRBcmtfR3Jqak4K" | base64 -d
      64_ooenwxoMLdTArk_GrjjN
  3. Enable the feature flag for this project's group, e.g. Feature.enable(:prefix_ci_build_tokens, Group.find(GROUP_ID))
    1. Re-run the job
      $ echo "Attempting to echo CI_JOB_TOKEN."
      Attempting to echo CI_JOB_TOKEN.
      $ echo $CI_JOB_TOKEN
      [MASKED]
      $ echo "Attempting to echo base64(CI_JOB_TOKEN)"
      Attempting to echo base64(CI_JOB_TOKEN)
      $ echo $CI_JOB_TOKEN | base64
      Z2xjYnQtNjRfVHRqejZTY3E2ck1Mc0I3SGY1MUIK
      % echo "Z2xjYnQtNjRfVHRqejZTY3E2ck1Mc0I3SGY1MUIK" | base64 -d
      glcbt-64_Ttjz6Scq6rMLsB7Hf51B
      1. Observe that the job token is still correctly masked
      2. Observe that when you base64 decode the value, it has the new prefix
      3. Observe that cURL still executes successfully; prefixed CI_JOB_TOKEN remains valid authentication/authorization /assign me

Related to #426137 (closed)

Edited by Nick Malcolm

Merge request reports