JWT CI/CD job tokens rejected after the instance token prefix changes

Problem

Running jobs that use a JWT CI/CD job token stop authenticating when the instance token prefix changes or when custom_prefix_for_all_token_types is disabled. The runner gets 403 on PUT /api/v4/jobs/:id and on trace updates, and the job stays running until it times out. API calls that use CI_JOB_TOKEN from inside the job fail too.

Disabling the feature flag is a likely rollback step while the feature is behind it. Doing that would cause this failure for every running job that uses a JWT job token. This should be fixed before custom_prefix_for_all_token_types is enabled by default.

Found during review of !258680 (comment 4099427912).

Steps to reproduce

  1. Enable jwt_ci_cd_job_token_enabled on a top-level group, enable custom_prefix_for_all_token_types, and set the instance token prefix to acme.
  2. Start a job that runs sleep 150 and then curl -H "JOB-TOKEN: $CI_JOB_TOKEN" "$CI_API_V4_URL/job".
  3. While the job sleeps, change the prefix to beta, or disable the feature flag.
  4. The runner log shows 403 on PUT /api/v4/jobs/:id and on the trace update, and the job stays running.

Cause

A token keeps the prefix it was created with, but two checks compare it against only the bare glcbt- prefix and the current instance prefix:

  • Ci::JobToken::Jwt.decode (lib/ci/job_token/jwt.rb)
  • Authn::Tokens::CiJobToken.prefix? (lib/authn/tokens/ci_job_token.rb), which guards the database-token fallback in Ci::AuthJobFinder and the job-token handling in Gitlab::Auth::AuthFinders

An acme-glcbt-… token fails both checks once the current prefix is beta or empty.

Not affected:

  • Setting a prefix for the first time or enabling the flag, because bare glcbt- tokens are always accepted
  • Database-backed job tokens, which never carry the instance prefix

Proposal

Match the instance prefix by its format instead of its current value, as Ci::Runner and Ci::Trigger already do with Authn::TokenField::PrefixHelper.optional_instance_prefix_pattern. Use the same pattern in both checks.

The JWT signature check still decides whether a token is accepted. The prefix only tells the code where the JWT starts, so accepting any well-formed instance prefix doesn't weaken authentication.

Authn::Tokens::StatelessAccessToken#matched_prefix (ee/lib/authn/tokens/stateless_access_token.rb) uses the same "bare or current prefix" logic and should get the same fix.

Acceptance criteria

  • A JWT job token created under one instance prefix still authenticates after the prefix changes, is cleared, or custom_prefix_for_all_token_types is disabled.
  • Authn::Tokens::CiJobToken.prefix? recognizes job tokens with any well-formed instance prefix.
  • StatelessAccessToken handles prefix changes the same way.
  • Specs cover the prefix-change and flag-disable cases.
Edited by 🤖 GitLab Bot 🤖