Stop answering 500 to a deploy token before it expires
What does this MR do and why?
A deploy token used on the API before it expires is sometimes answered 500 Internal Server Error. The issue shows it on a generic package download with a token created through the API with expires_at ten minutes ahead, and the exception is NoMethodError: undefined method 'blocked_pending_approval?' for an instance of DeployToken at lib/gitlab/auth/user_access_denied_reason.rb:42. Two things go wrong one after the other:
- Two answers to "has this token expired?" Authentication finds a deploy token through the
DeployToken.activescope, which comparesexpires_atwith the current time: the database'sNOW()when the issue was reported, and the application'sTime.currentsince !263037 (merged).API::APIGuard::HelperMethods#find_current_user!then asksuser.can?(:access_api), whichGlobalPolicyrefuses for adeactivated?actor, andDeployToken#active?decided expiry with a private override ofExpirable#expired?that comparesexpires_atwithDate.today, the date of the host's clock in its local time zone. On a host whose local date is already ahead of the UTC date, a token that expires before the UTC day ends is found by the scope and then refused as deactivated, before it expires. In the issue, the 500 came at 23:32 UTC, while the scope still found the token (the refusal message is only built for a token authentication found), and a fresh token worked at 00:13 UTC. The issue's own logs show such a host: Workhorse logged the same request (correlation ID01KS19AYJ3B2K1NK6NXSAW6VRR) at2026-05-20T09:32:32+10:00, so the host ran on UTC+10 and its date was already 2026-05-20. At 00:13 UTC the UTC date was 2026-05-20 too, so a token the scope still found could not carry an earlier date, which is why the fresh one worked. - A refusal message that only a user can produce. The refusal goes through
forbidden!(api_access_denied_message(user)), andGitlab::Auth::UserAccessDeniedReason#rejection_typereads account states only aUserhas, starting withblocked_pending_approval?. A deploy token has none of them, so the refusal itself raises and the request answers 500 instead of 403.
The commits:
- Refuse an expired deploy token with 403 instead of 500:
UserAccessDeniedReasonanswers a deploy token with reasons of its own,This deploy token has been revoked.orThis deploy token has expired., and any other refusal of a deploy token with the generic message. Unit specs for each reason, and a request spec on the generic package download where the lookup finds the token andDeployToken#active?answersfalse: onmasterthat answers 500 with the issue'sNoMethodError, on this commit 403 with the expired message. It carries theChangelog: fixedtrailer, on the first commit as the changelog guidelines ask. - Cover the revoked and expired fields of the deploy token entity:
API::Entities::DeployTokenexposesDeployToken#expired?asexpired, and no spec pinned its value. This pins it for an active token, a revoked one, one that expired on an earlier day and one with no expiry. It passes onmasterand on every commit of this MR. - Expire a deploy token at the time in its expires_at: removes the private override, so
Expirable#expired?(expires_at <= Time.current) applies and a deploy token expires at the instant inexpires_at, when the scope stops finding it. Specs: on a host whose date is the UTC date, a token that expired at the start of the UTC day is no longer active and the entity reportsexpired: true; a token that expires at the end of the UTC day is still active when the host's date is already the next day; and a token that expires in an hour still downloads a package when the host's date is already past the date it expires on. Each stubsDate.today, so none depends on the zone of the machine running it, and all four fail on the previous commit. The request spec gives its token an hour rather than the rest of the UTC day because the lookup comparesexpires_atwithTime.current, throughExpirable.not_expired, and the spec stubsDate.todayand notTime.current, so a fixed instant of the day would fail when the spec runs in the day's last moments. It also carriesChangelog: fixed, since theexpiredfield of the deploy tokens API changes. - Ask a refused deploy token whether it has expired, from review:
UserAccessDeniedReasonnow asksexpired?, public onDeployTokensince commit 3, instead ofdeactivated?. It is the same answer today, and a token refused for any other reason gets the generic message rather than the expired one. The request spec from commit 1 stubsexpired?instead ofactive?, and a unit example covers a token that is inactive without having been revoked or expired. - Drop no-op date stubs and note the revoked reason, from review: the four examples of commit 3 stub
Date.today, which the expiry check no longer reads once the override is gone, so the stubs go and each context is named after the instant its token expires at. A comment inUserAccessDeniedReasonsays that the revoked branch is defensive, since every lookup that can hand it a deploy token goes throughDeployToken.active, which leaves revoked tokens out, and that a token can still expire between the lookup and the check.
Two visible changes beyond the 500, both making Ruby agree with what authentication already does: the expired field of the deploy tokens API is now true from the instant the token stops authenticating, instead of from the next date of the host's clock, and a token the scope finds is no longer refused as deactivated before it expires. For a token whose expiry is a date, that is what the deploy tokens page already says: it expires at 00:00 UTC on that date.
The scope itself changed in !263037 (merged), the follow-up for #633608 (closed) agreed in review: it is now built from Expirable.not_expired, so it compares expires_at with the application's Time.current, the clock Expirable#expired? reads, and stops finding a token at the instant expired? starts reporting it expired (the scope's > against expired?'s <=). With commit 3, the lookup and DeployToken#active? agree on both the clock and the instant. They still read the clock at two moments of one request, so a token that expires between the lookup and the access check is found and then refused, and with commit 1 that request answers 403 with the expired message rather than 500.
References
Closes #600629
Screenshots or screen recordings
Not a UI change. The generic package download with a deploy token that expires at the end of the UTC day, from a host whose local date is already the next day:
| Before | After | |
|---|---|---|
| Response | 500 Internal Server Error |
200 OK |
Specs I ran
Except for the last bullet, on top of master at 19b4fa243a2; all in the test environment, with bundle exec rspec. Where it says Kiritimati, the run had TZ=Pacific/Kiritimati (UTC+14), so the host's date was already the next day (those runs were between 17:15 and 17:50 UTC, and the last bullet's between 16:34 and 16:41 UTC); where it says the host's zone, it was Europe/Madrid, whose date was the UTC date.
- A throwaway request spec (not part of this MR) that creates a deploy token with
read_package_registryandexpires_atat the end of the UTC day and downloads a generic package file with it, nothing stubbed, under Kiritimati:master:500,NoMethodError (undefined method 'blocked_pending_approval?' for an instance of DeployToken)fromlib/gitlab/auth/user_access_denied_reason.rb:42, the issue's exception.DeployToken.new(expires_at: Time.current.end_of_day).active?wasfalse.- Commit 1:
403,{"message":"403 Forbidden - This deploy token has expired."}. - Commit 3:
200, andactive?wastrue.
- Commit 1's specs on
master's code, under Kiritimati: 14 examples, 5 failures, the probe and the four new examples: the threeUserAccessDeniedReasonones (NoMethodError) and the request one (500instead of403). On commit 1: 14 examples, the probe alone failing. - Commit 2's spec on
master's code and on commit 2: 4 examples, 0 failures each time. - Commit 3's specs (
spec/models/deploy_token_spec.rb,spec/lib/api/entities/deploy_token_spec.rband the two new generic packages contexts) on commit 2's code: 100 examples, 4 failures, under Kiritimati and again in the host's zone, the four new examples each time; the request one answers403with the expired message. On commit 3: 0 failures in both zones, and under Kiritimati the probe passes too. - On commit 3, in the host's zone:
spec/requests/api/generic_packages_spec.rb: 418 examples, 0 failures, 4 pending.spec/lib/gitlab/auth/auth_finders_spec.rbandspec/lib/gitlab/auth_spec.rb: 560 examples, 0 failures.spec/models/deploy_token_spec.rb,spec/lib/api/entities/deploy_token_spec.rb,spec/lib/gitlab/auth/user_access_denied_reason_spec.rb,spec/models/concerns/expirable_spec.rb,spec/finders/deploy_tokens/tokens_finder_spec.rb,spec/lib/gitlab/conan_token_spec.rb,spec/workers/deploy_tokens/expiring_worker_spec.rb,spec/requests/api/deploy_tokens_spec.rbandspec/models/project_auto_devops_spec.rb: 344 examples, 0 failures, 10 pending.- The
#gitlab_deploy_tokengroups ofspec/models/project_spec.rbandspec/models/group_spec.rb: 14 examples, 0 failures. - Four controller specs that read deploy tokens and
ee/spec/requests/custom_roles/regular/manage_deploy_tokens/request_spec.rb: 212 examples, 159 failures, every oneLoadError: cannot load such file -- sasswhile rendering a page in this environment. The same 159 examples fail the same way onmaster.
- Commit 4, from review, in the host's zone, on
user_access_denied_reason_spec.rband the two generic packages contexts: commit 4's code with commit 3's specs, 14 examples, 1 failure (the request spec that stubsactive?now gets the generic message); commit 3's code with commit 4's specs, 15 examples, 1 failure (the new unit example gets the expired message); commit 4, 15 examples, 0 failures. Thenuser_access_denied_reason_spec.rb,generic_packages_spec.rb,api_guard_spec.rb,spec/lib/api/helpers/authentication_spec.rb,spec/models/deploy_token_spec.rbandspec/lib/api/entities/deploy_token_spec.rbon commit 4: 567 examples, 0 failures, 4 pending. - RuboCop on the Ruby files of each commit: no offenses. Both commit message linters (
scripts/lint/commit_linter.rband the Danger one) on each commit: no problems.git diff --check: clean. - After rebasing onto
masterat1b9d137a11d, which has !263037 (merged), on commit 4: the model, entity andUserAccessDeniedReasonspecs, 113 examples, 0 failures, and the GET authentication context ofgeneric_packages_spec.rb, 78 examples, 0 failures, in the host's zone and again under Kiritimati; the wholegeneric_packages_spec.rb, in the host's zone, 418 examples, 0 failures, 4 pending. Commit 3's specs on commit 2's code, in the host's zone: 102 examples, 4 failures, the four new examples. RuboCop, both commit message linters andgit diff --checkon each rebased commit: clean.
How to set up and validate locally
The bug needs a host clock whose local date is ahead of the UTC date. Pacific/Kiritimati is UTC+14, so its date is a day ahead from 10:00 UTC onwards.
-
From 10:00 UTC, start a Rails console under that time zone, from the
gitlabdirectory of the GDK:TZ=Pacific/Kiritimati bin/rails console -
Ask a token that expires at the end of the UTC day whether it is active:
DeployToken.new(expires_at: Time.current.end_of_day).active?It answers
falseonmasterandtruewith this MR. -
To see the 500 itself, run the GDK's Rails under the same
TZ, create a project deploy token with theread_package_registryscope through the API with anexpires_atbefore the end of the UTC day, and download a generic package file with it:curl --header "Deploy-Token: <token>" "http://gdk.test:3000/api/v4/projects/<id>/packages/generic/<name>/<version>/<file>"On
masterthat answers500 Internal Server Error; with this MR, the file.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.