Rotate leaked PATs into Secrets Manager from vulnerabilities

https://docs.google.com/document/d/1LO7Q64jwApUnj3qxcnF-6e3NUHovrbj_us6TvbN76a0/edit?tab=t.u9ragcxjvc6j

Overview

  1. Detect a GLPAT using Pipeline SD
  2. Validate the GLPAT is active using Validity Checks
  3. Trigger rotation flow via this proposed design
  4. Rotate the active GLPAT
  5. Look into using this endpoint for this, requires Admin privileges
  6. Grab the newly created GLPAT value and store in the Secrets Manager of your choosing requires more thought on variable naming, path at which to store it
  7. Generate an MR to replace the leaked value with the new value that was stored in the secrets manager

References

  1. Challenge doc - https://docs.google.com/document/d/1LO7Q64jwApUnj3qxcnF-6e3NUHovrbj_us6TvbN76a0/edit?tab=t.u9ragcxjvc6j
  2. Designs - https://www.figma.com/board/fou0HGe2HU9gMqrbQ0ucSA/Flowchart---Challenge--10?node-id=0-1&p=f&t=ceUy6e9m1rjPhkht-0

Testing

  • GraphQL-only rework at 926ded5b: REST is removed, and the UI calls the projectSecretCreateFromLeakedToken mutation. Locally, the new request spec (25 examples, real OpenBao), both service specs, and the vulnerability actions, modal and dropdown Jest specs pass. The REST and GDK bullets below were run against the earlier REST endpoint. On GDK at f6c7612f I ran the mutation with curl against a fresh seed:
    • Owner rotating a legacy leaked token: success. The old token is revoked, the replacement is active with previous_personal_access_token_id set and the same expiry, the secret is stored, the finding is inactive, and the response has no token value.
    • A fine-grained leaked token: success. The replacement keeps the same granular scopes.
    • Repeating on the same vulnerability: refused with "Token already revoked". There's no second replacement, and the first replacement stays active.
    • Refused, with the leaked token still active and no secret stored:
      • a guest, and an outsider (generic access error)
      • a fine-grained caller with only create_secret (missing Personal Access Token: Rotate)
      • the flag turned off
      • a duplicate secret name
    • Not re-checked: the UI click path on the GraphQL mutation.
  • CI: pipeline 2998612822 passed on 9b778a7a. It runs the service specs against a real OpenBao, the REST and GraphQL request specs, and the three Jest specs. 299d6a5b adds fine-grained token specs; locally both service spec files pass (26 examples).
  • REST on Caproni (an earlier commit of this branch, real OpenBao): unauthenticated 401, outsider 404, guest 403, missing vulnerability 404, invalid secret name 422, owner 201 with metadata only, repeat call 422 "Token already revoked".
  • UI on GDK at 9b778a7a, as a non-admin group owner: Fix with AI > Rotate and fix on a secret detection vulnerability. The leaked token was revoked, a replacement was created, the secret was stored, and the finding's token status changed from active to inactive. Duo didn't start because there was no runner; the warning alert says so and links to the stored secret.
  • Fine-grained token on GDK, through REST: 201. The leaked token was revoked, and the replacement is fine-grained with the same permissions (selected_memberships, project namespace, create_work_item and read_work_item). The finding changed to inactive.
  • Fine-grained tokens enforced on GDK, legacy leaked token, through REST: 422 "Rotation of legacy personal access tokens is disabled. Use a fine-grained token instead." Nothing was rotated or stored, and the leaked token stayed active.
  • Not tested: the full Duo loop that opens the fix MR.

Risk

High per mr-harness (it creates and revokes credentials), but off by default: everything is behind the beta flag secrets_manager_pat_rotation (default_enabled: false). There are no migrations and no changes to existing code paths. The feature has one entry point, the GraphQL mutation projectSecretCreateFromLeakedToken, and no REST endpoint.

Known behaviour reviewers should check
  • If storing the new token in OpenBao fails after rotation, the new token is revoked too, so no unreachable live credential remains. The leaked token is already revoked by then, so whatever used it breaks until someone creates a new token.
  • When an instance enforces fine-grained tokens, rotating a legacy token is refused. The mutation returns the error, but the leaked token stays active and must be revoked by hand.
  • The fix merge request that Duo opens is not linked to the vulnerability, so the vulnerability's related merge requests stay empty. Linking it in code is tracked in #637386.
  • The replacement token keeps the leaked token's name, so the owner's token list shows an entry identical to the leaked one. Marking it as rotated is tracked in #638139.
  • The server resolves the leaked token from the vulnerability finding, so the client never sends a token value. The user's access to the vulnerability is checked before the lookup.

How to revert

Turn off secrets_manager_pat_rotation, or revert this MR. Neither undoes rotations already done: revoked tokens stay revoked, and stored secrets stay in Secrets Manager.

Edited by Mark Mishaev

Merge request reports

Loading
Loading