Add Secrets Manager direct API access for non-CI/CD workloads

Erick Bajaorequested to merge
eb-direct-access-secret into master

What does this MR do and why?

Adds a beta REST endpoint that lets non-CI/CD workloads read GitLab Secrets Manager secrets, closing the gap where only CI/CD jobs (via Runner JWT) and the UI (via the Rails GraphQL proxy) could reach secrets.

POST /api/v4/projects/:id/secrets_manager/access_token
POST /api/v4/groups/:id/secrets_manager/access_token

The endpoint mints a short-lived JWT scoped to the caller and returns the OpenBao connection details. The client then reads secrets directly from OpenBao, the same direct-access pattern Runners already use. The response is shaped like external-secrets.io/v1.VaultProvider so clients (External Secrets Operator, Terraform, scripts) can consume it with standard Vault-compatible tooling.

The endpoint mints a token; it never reads a secret value. The value read happens later, client-side, against OpenBao.

Example

Request (no body). The token can be a regular PAT, a service account token, or a group/project access token, as long as the caller is Reporter or above on the namespace. A standalone granular token also works if it carries the create_secrets_manager_api_jwt permission. Service accounts use a regular PAT (granular tokens are not extended to service accounts yet).

curl --request POST \
  --header "PRIVATE-TOKEN: <access-token>" \
  "https://gitlab.example.com/api/v4/projects/99/secrets_manager/access_token"

Response:

{
  "expires_at": "2026-06-12T10:35:00Z",
  "provider": {
    "vault": {
      "server": "https://openbao.example.com",
      "namespace": "org_5/ns_42/project_99",
      "path": "secrets/kv",
      "version": "v2",
      "auth": {
        "jwt": {
          "path": "api_jwt",
          "role": "all_api",
          "token": "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9..."
        }
      }
    }
  }
}

The client takes provider.vault and logs in to OpenBao with the JWT (auth.jwt), then reads from the KV path. The response is shaped like external-secrets.io/v1.VaultProvider, so it maps onto a Vault provider config with the token inlined.

Authorization model and why Reporter access to the endpoint is fine

Access is enforced in two independent layers:

  1. Minting a token (this endpoint) is gated by a new, dedicated permission create_secrets_manager_api_jwt, granted to Reporter and above on an SM-enabled namespace.
  2. Reading a secret value is gated separately in OpenBao by the per-principal read_value permission. The minted JWT, presented to the new api_jwt mount, only attaches a read-only policy for secrets the principal was explicitly granted read_value on.

Why Reporter access to minting is fine: the token is useless on its own. A Reporter (or any caller) with no read_value grant gets a token whose OpenBao policies do not exist, so every read is denied. Minting gives them nothing. The real check is read_value, done in OpenBao. This is covered end to end in ee/spec/requests/secrets_management/api_secret_access_spec.rb (granted reads work; metadata-only, revoked, and ungranted callers are denied).

We allow minting for Reporter and above because read_value can be granted to any Reporter+ member, so the mint has to be open to the same people. If we made it stricter, a Developer who was granted read_value could not get a token to use it. We also cannot check "has read_value somewhere" when minting, because read_value lives in OpenBao, not the GitLab database. So the people who can mint are the same ones who already have :read_project_secrets.

Permission naming

The permission is named for what the endpoint does. It creates a JWT for the api_jwt OpenBao role, so it is create_secrets_manager_api_jwt, not read_project_secrets (which would imply reading values the endpoint never performs). The api_jwt suffix also distinguishes it from the other Secrets Manager JWTs (pipeline_jwt, user_jwt). It is registered as a granular token (GPAT) permission and assignable on project and group boundaries.

Maturity and rollout

  • Ships as beta (route_setting :lifecycle, :beta), aligned with SM's Open Beta.
  • Behind the secrets_manager_api_access feature flag, disabled by default.
  • Excluded from the OpenAPI spec until general availability.

Databases

No migrations. No schema or query changes. The new OpenBao api_jwt mount and all_api CEL role are additive and provisioned alongside the existing pipeline_jwt and user_jwt mounts. Existing SM-enabled namespaces are backfilled separately (see References).

Backward compatibility

  • The endpoint is off by default behind the feature flag.
  • The OpenBao change is additive; existing pipeline_jwt and user_jwt mounts and roles are untouched.
  • The existing read secret-permission action is kept as a deprecated alias of read_metadata (accepted on input, still emitted on readback), so the frontend can migrate on its own clock. No data migration.

Testing

  • Unit: permission model (path-aware capability mapping, read_value validation), JWT classes (TTL, api scope, auth_via).
  • Integration (:gitlab_secrets_manager, against a real OpenBao): CEL boundaries for the api_jwt mount (project and group), provisioning of the mount and role, and the end-to-end read_value value-gating behaviour.
  • Request: the endpoint contract, authorization (Reporter+ allowed, Guest/non-member denied, FF off, inactive manager), and the granular-token shared example.
  • Policy specs for create_secrets_manager_api_jwt on project and group.

Documentation

  • New API reference page and a "non-CI/CD access" guide.
  • Needs a Technical Writer review before merge.

Deferred (tracked separately)

  • "Read value" permission UI and GraphQL exposure (so users can grant it).
  • Backfill of the api_jwt mount/role into existing SM-enabled namespaces.
  • QA end-to-end spec for the deployed read path.
  • Workload identity federation (exchange external OIDC for an SM JWT).
  • Usage tracking and billing.

References

Edited by Erick Bajao

Merge request reports