Count CI job tokens sent as basic auth or bearer against the job user (!259720) · Merge requests · GitLab.org / GitLab

What does this MR do and why?

CI job tokens sent as HTTP basic auth (username gitlab-ci-token, password $CI_JOB_TOKEN) or as Authorization: Bearer $CI_JOB_TOKEN are now resolved by the rack rate limiter. These requests count against the job user's per-user limits. Before, they counted against the unauthenticated per-IP limit (60 an hour on GitLab.com).

The token is valid and the endpoint accepts it. But the rate limiter runs before a route is picked, so it never resolved the token. CI jobs share runner egress IPs, so many jobs share one IP's 60 requests.

One day of observe-mode data (table in the issue) shows what enforcement would block:

About 60k successful Terraform state requests a day: 42.2k on /terraform/state/:name and 18.4k on /terraform/state/:name/lock. A job that takes the lock and then gets a 429 on the upload or unlock leaves the state locked until someone force-unlocks it.
2.9k successful GET /api/v4/job requests a day with a bearer job token.

The brownout on 7 October enforces the unauthenticated per-IP limit. The brownout analysis marks Terraform state as blocking it, so this needs to reach production before 7 October.

How it works

Both changes are in lib/gitlab/auth/request_authenticator.rb, plus a prefix check in lib/gitlab/auth/auth_finders.rb.

Basic auth: route_authentication_setting returns job_token_allowed: %i[request basic_auth] on API paths instead of true. find_user_from_job_token only tries basic auth when the setting includes :basic_auth, and true never did. The rate limiter does not know which route will handle the request, so it accepts every form an API route takes a job token in. Header and query job tokens keep working.
Bearer: find_user_from_any_authentication_method now tries find_user_from_job_bearer_token before the OAuth and personal access token finder. This is the same order API::APIGuard uses.
Cost: find_user_from_job_bearer_token now checks the glcbt- prefix first, so an OAuth or personal access token bearer request skips both the job lookup and the parse of the JWT signing key (Ci::JobToken::Jwt.decode parses it on every call). This also saves API::APIGuard the same work. Basic auth with a username other than gitlab-ci-token returns before a lookup.
Failures stay safe: an unknown, expired or not-running job token resolves to no user, so the request stays anonymous, as today. gitlab-ci-token:<personal access token> still resolves to the token's user.
Only /api/ paths change. Web controllers, the GraphQL channel and GraphQL's own finder (which does not accept job tokens) are not affected.

Behaviour change outside GitLab.com

RequestAuthenticator also feeds the existing Rack::Attack throttles on every instance. Job-token basic-auth and bearer requests on /api/ move from the unauthenticated API throttle (per IP) to the authenticated API throttle (per user). Header job tokens already behave this way, so this removes an inconsistency.

API::Internal::Workhorse#request_authenticated? also calls find_authenticated_requester([:api]) for /api/v4/internal/workhorse/authorize_upload. A job token sent as basic auth or as a bearer token now passes that check too, as a JOB-TOKEN header already does. The endpoint only returns a TempPath.

Ci::AuthJobFinder now also runs at the rack layer for these tokens, so each request writes the successful job token auth log line twice, as JOB-TOKEN requests already do. For Terraform state that is about 60k extra lines a day.

Job token audit events

user_authenticated_using_job_token (streamed, not saved to the database) is emitted inside find_user_from_job_token, so every rate-limiter lookup emitted one too. Before this MR a JOB-TOKEN header request already emitted 4 per request (the endpoint, plus Rack::Attack and the Labkit middleware), and resolving basic-auth job tokens at the rack layer would have taken basic auth from 1 to 4. This is the mechanism in #631338.

EE::Gitlab::Auth::RequestAuthenticator now makes audit_job_token_authentication a no-op, as proposed on that issue. Every rate-limiter lookup goes through RequestAuthenticator, while real API authentication goes through API::APIGuard, which still emits the event. Measured per request: JOB-TOKEN header 4 → 1, basic auth 1 → 1 (instead of 4), bearer 0 → 0. This applies to every EE instance (GitLab.com, Dedicated and self-managed), since both the event and the override are EE code. Workhorse's authorize_upload pre-check also stops emitting its own copy; the upload request it precedes is still audited by the API guard. The bearer form was never audited, which is unchanged.

The commit has a Changelog: fixed trailer. Need an authentication review.

On GitLab.com, the package registries that also accept job tokens as basic auth (PyPI, Composer, generic packages) carry the rate-limit bypass header. Only Terraform state and /api/v4/job are affected there.

References

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited Oct 06, 2026 by Ashwin S

Count CI job tokens sent as basic auth or bearer against the job user