Switch CI_JOB_TOKEN to be JWT

Deprecation Summary

The Job Token format is being moved from a legacy string to a JWT format to support various requirements in the product. As a result, the Runner with AWS Fargate Custom Executor driver on version 0.5.0 or earlier is incompatible with the new CI JWT job token format.

Documentation

Product Usage

The job token is generated for every job and is used to access the GitLab API. The token is originally as encoded string and will be switching to the JWT format to support various product requirements inlcuding:

  • Embed Composite Identity inside CI_JOB_TOKEN
  • Encoding granular permissions for CI_JOB_TOKEN
  • Add Cell CI to CI_JOB_TOKEN JWT payload (#489855 - closed)
  • CI now maintains 2 versions of the CI_JOB_TOKEN and we need to remove the maintenance burden.
    1. Existing/legacy encrypted string. This is generated when the job goes into pending state and persisted in the database. Then, when the job completes we remove it.
    2. New JWT implementation. This is generated on the fly when we send the token to the runner.

Known Issues

  1. AWS Fargate Driver Compatibility: GitLab Runner's AWS Fargate Driver versions 0.5.0 and earlier are incompatible with the JWT standard. If you use AWS Fargate custom executor driver, please upgrade to version 0.5.1 or later. Migration instructions are available in our documentation.
  2. Base64 Encoding: The JWT standard tokens are longer, which breaks the echo $CI_JOB_TOKEN | base64 command if used in your CI/CD configurations. Please update to echo $CI_JOB_TOKEN | base64 -w0 instead.

Breaking Change?

Does this deprecation contain a breaking change? Yes

Affected Customers

Who is affected by this deprecation: GitLab.com users, Self-managed users, or Dedicated users? (choose all that apply)

  • GitLab.com
  • Self-managed
  • Dedicated

What pricing tiers are impacted?

  • GitLab Free
  • GitLab Premium
  • GitLab Ultimate

Deprecation Milestone

This deprecation will be announced in milestone: 17.9

Planned Removal Milestone

The feature / functionality will be removed in milestone: 19.0

Links

Checklists

Timeline

Rollout Plan

Development Tasks

Communication Plan

DRI Product Manager: @jrandazzo

Internal communication
External Communication

Approvals

  • Product Manager @PM
  • Engineering Manager @EM
  • Senior Engineering Manager / Director @senior-eng-leader
  • Group / Director of Product Management @senior-product-leader
  • Product / Eng Leaders in the CPO or CTO organizations, as applicable (optional - depends on scope of change)

References

Original issue description

We have implemented a JWT version of CI_JOB_TOKEN and tried to roll it out in [Feature flag] Rollout of `ci_job_token_jwt` (#497392).

We had to disable the feature flag because:

  1. The new JWT broke an existing implementation of AWS Fargate runner custom executor. This was then fixed but it now requires affected customers to upgrade to v0.5.1.
  2. Other customers also reported that the much longer JWT broke their use of echo $CI_JOB_TOKEN | base64 pipe command which now prints the value across multiple columns by default. This latter problem can be fixed by using | base64 -w0 instead.

Requirements on JWT

The switch to use JWT is important for a number of features:

  • Embed Composite Identity inside CI_JOB_TOKEN
  • Encoding granular permissions for CI_JOB_TOKEN
  • Add Cell CI to CI_JOB_TOKEN JWT payload (#489855 - closed)
  • CI now maintains 2 versions of the CI_JOB_TOKEN and we need to remove the maintenance burden.
    1. Existing/legacy encrypted string. This is generated when the job goes into pending state and persisted in the database. Then, when the job completes we remove it.
    2. New JWT implementation. This is generated on the fly when we send the token to the runner.

Proposal

Issue breaking change warning now and rollout [Feature flag] Rollout of `ci_job_token_jwt` (#497392) in %18.0

Edited by Ajay Thomas