Draft: Route /jwt/auth by container registry scopes and Basic credential
What does this MR do and why?
Stacked on !1383 (merged), which owns the routable-token grammar this MR's credential path decodes through, and which is itself stacked on !1384 (merged), where
decodeRoutableToken, the one call this MR's credential path makes, lives. Merge both first, bottom up; this then retargets tomain.
Implements the routing algorithm for /jwt/auth, the un-routable response and routable tokens from the Container Registry Routing Service design document.
Every container registry token request reaches the Legacy Cell today: the session_token ruleset ends in an unconditional proxy, and none of the classify rules above it can match a Docker client, which authenticates with Authorization: Basic. Two things in the request can name the cell that holds the repository, and this MR uses both, in order:
- The
scopeparameters. Write scopes decide when the request has any, otherwise the pull scopes do. They are deduplicated to distinct top-level namespaces, capped (GITLAB_JWT_AUTH_MAX_DISTINCT_ROUTES, default 10, checked before any RPC) and resolved concurrently, one single-claimClassifyeach. One cell means proxy to it; two or more different cells, or over the cap, means400 UNSUPPORTED, answered here without contacting a cell. - The routable token in the
Authorization: Basicheader, when the scopes did not decide.docker logincarries a credential and no repository scope at all, and so does any request whose scope names a namespace no cell owns. The decodedcgoes out as the scalarCELL_IDclaim the shared ruleset's routable-token rules already use, so the cell reads the samex-gitlab-http-router-rule-typewhichever door the request came through. Both the versioned routable form and the CI job token's JWT form decode.
Anything neither decides, including any Classify failure, falls to the JSON rule engine and is answered exactly as on main. The two 400s are never rescued by the credential, and a scope-path Classify failure deliberately does not consult it: that lookup needs the same unhealthy service, and a failure is not a conclusion. Nothing here verifies the token; the router emits a routing hint and the destination cell re-authenticates.
src/jwt_auth/decision.ts owns the algorithm end to end as one pure injectable function returning the decision and its telemetry as separate values, src/handlers/jwt_auth_handler.ts owns only the HTTP, and src/jwt_auth/credential.ts owns the HTTP Basic half alone. The one file this touches outside /jwt/auth is src/routable_token.ts, where RoutableTokenEncoding's routable member becomes versioned: the encoding is reported now, and the family name being also one member's name reads wrong against unroutable, since jwt is a routable form too. The pair is reported rather than collapsed because the two decoders are not interchangeable - versioned is pinned to one payload version, jwt is not - so a rise in unroutable stays attributable: a v02 mint reads as versioned falling. It is not a per-credential signal: a credential made routable (gitlab-com/gl-infra/tenant-scale/cells-infrastructure/team#917) takes one of these two shapes, not a third.
Rollout. Gated on GITLAB_JWT_AUTH_ROUTING_ENABLED, a Flagship boolean of its own defaulting to off and read per request, so merging changes nothing until the flag is turned on, and dark launch, rollout and rollback need no deploy. Nothing has to happen before merge: an unevaluable flag returns the false default. Order is merge and deploy, create the flag, enable in gstg, roll out in gprd, written up in docs/path-based-routing.md. The decision log line carries decidedBy, credentialEncoding (versioned, jwt, unroutable or absent) and distinctRouteCount, the numbers the rollout reads. Never logged: the token, any substring of it, or the decoded u/o/p/g.
Known and accepted: one extra unauthenticated Topology Service call per undecided request, cache-keyed on a body carrying an unverified cell id, so crafted tokens yield unbounded distinct keys. This is parity with the rule engine's generic Bearer PAT rule, which already does exactly this on every /api/* request with no cap and no verification.
Five commits, each green on its own. npm test 36 files / 465 tests at the base → 41 / 593. tsc --noEmit, eslint ., prettier --check and npm run jsonnet-test clean.
References
- Related issue: gitlab-com/gl-infra/tenant-scale/cells-infrastructure/team#881 (closed)
- Epic: gitlab-com/gl-infra/tenant-scale/cells-infrastructure#54
- Design document (unmerged): gitlab-com/content-sites/handbook!21159 (merged), Container Registry Routing Service
- Depends on !1383 (merged), the routable-token grammar and the drift check that keeps the rules and the module in step
- Depends on !1384 (merged), below !1383 (merged), and the MR that actually supplies
decodeRoutableToken; reviewable and mergeable on its own
How to set up and validate locally
npm test covers every outcome below without a running stack. To drive it by hand on a GDK:
Run the router with the flags on
Two flags, and they go in two different places. GDK passes GITLAB_PATH_BASED_ROUTING_MODE as a CLI --var, and a CLI --var beats .dev.vars (getBindings2 spreads the CLI bindings last), so that one has to come from GDK's own setting. The rollout flag and the log level are not passed by GDK, so .dev.vars is enough for those.
# <gdk>/gdk.yml
gitlab_http_router:
path_based_routing_mode: full# <gdk>/gitlab-http-router/.dev.vars (gitignored)
GITLAB_JWT_AUTH_ROUTING_ENABLED=true
LOGGING_LEVEL=debuggdk reconfigure && gdk restart gitlab-http-routerBoth are load-bearing: the rollout flag defaults to off, and GDK defaults the mode to legacy, in which /jwt/auth is not registered at all and every request below falls through to the ruleset unchanged.
gdk tail gitlab-http-routerThe log is the only oracle, since GDK's cell 1 address is the same 127.0.0.1:3333 as GITLAB_PROXY_HOST and a routed request is byte-identical to a fallback. Check the setup took with:
{ msg: 'GITLAB_PATH_BASED_ROUTING_MODE resolved', mode: 'full' }mode: 'legacy' there means the route never ran. Note this line only appears from the second request onward: the mode is logged at debug on a singleton logger whose level is raised by LOGGING_LEVEL inside a handler, so the first request through is still at the default level.
Requests
Each is one command. GDK configures a single cell, id 1, and an empty claim store.
Routed by the credential, JWT form (a CI job token).
curl -u 'someone:e30.eyJjIjoiMSJ9.sig' 'http://127.0.0.1:3000/jwt/auth?service=container_registry&offline_token=true&account=someone'{ msg: '/jwt/auth routing decision', decidingScopeKind: 'none', scopeCount: 0,
distinctRouteCount: 0, outcome: 'proxied', decidedBy: 'credential', credentialEncoding: 'jwt' }Routed by the credential, versioned routable form (a PAT).
curl -u 'someone:glpat-xxxxxxxxxxxxxxxYzoxAw.01.06abcdefg' 'http://127.0.0.1:3000/jwt/auth?service=container_registry&offline_token=true&account=someone'{ msg: '/jwt/auth routing decision', decidingScopeKind: 'none', scopeCount: 0,
distinctRouteCount: 0, outcome: 'proxied', decidedBy: 'credential', credentialEncoding: 'versioned' }No credential.
curl 'http://127.0.0.1:3000/jwt/auth?service=container_registry&offline_token=true'{ msg: '/jwt/auth routing decision', decidingScopeKind: 'none', scopeCount: 0,
distinctRouteCount: 0, outcome: 'rule_engine_fallback', credentialEncoding: 'absent' }Credential that names no cell.
curl -u 'someone:gldt-xxxxxxxxxxxxxxxxxxxx' 'http://127.0.0.1:3000/jwt/auth?service=container_registry&offline_token=true&account=someone'{ msg: '/jwt/auth routing decision', decidingScopeKind: 'none', scopeCount: 0,
distinctRouteCount: 0, outcome: 'rule_engine_fallback', credentialEncoding: 'unroutable' }Credential that decodes, naming a cell this GDK does not configure. Keeps its real encoding rather than reporting unroutable.
curl -u 'someone:e30.eyJjIjoiMiJ9.sig' 'http://127.0.0.1:3000/jwt/auth?service=container_registry&offline_token=true&account=someone'{ msg: '/jwt/auth routing decision', decidingScopeKind: 'none', scopeCount: 0,
distinctRouteCount: 0, outcome: 'rule_engine_fallback', credentialEncoding: 'jwt' }Write scopes decide, pull scopes are context. Two scopes, one deciding namespace.
curl 'http://127.0.0.1:3000/jwt/auth?service=container_registry&scope=repository:aaa/p/i:pull&scope=repository:bbb/p/i:push'{ msg: '/jwt/auth routing decision', decidingScopeKind: 'write', scopeCount: 2,
distinctRouteCount: 1, outcome: 'rule_engine_fallback', credentialEncoding: 'absent' }Over the cap — a real 400, answered without contacting a cell.
curl -i 'http://127.0.0.1:3000/jwt/auth?service=container_registry&scope=repository:ns1/p/i:pull&scope=repository:ns2/p/i:pull&scope=repository:ns3/p/i:pull&scope=repository:ns4/p/i:pull&scope=repository:ns5/p/i:pull&scope=repository:ns6/p/i:pull&scope=repository:ns7/p/i:pull&scope=repository:ns8/p/i:pull&scope=repository:ns9/p/i:pull&scope=repository:ns10/p/i:pull&scope=repository:ns11/p/i:pull'HTTP/1.1 400 Bad Request
{"errors":[{"code":"UNSUPPORTED","message":"a single token cannot cover more than 10 top-level namespaces","detail":{"distinctRouteCount":11,"cap":10}}]}
{ msg: '/jwt/auth routing decision', decidingScopeKind: 'pull', scopeCount: 11,
distinctRouteCount: 11, outcome: 'unsupported_cap' }Anything that is not a container registry token request is untouched.
curl 'http://127.0.0.1:3000/jwt/auth?service=dependency_proxy&scope=repository:aaa/p/i:pull'no '/jwt/auth routing decision' line at all - the handler returns before loggingdecidedBy: 'scope', a redirectRoute match and unsupported_mixed_cell need claim data GDK does not have: the claim store is empty and only one cell is configured, so every route lookup correctly answers "no cell". All three are covered by npm test.
To undo: delete .dev.vars and gdk restart gitlab-http-router.
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.