<!--IssueSummary start--> <details> <summary> Everyone can contribute. [Help move this issue forward](https://handbook.gitlab.com/handbook/marketing/developer-relations/engineering/community-contributors-workflows/#contributor-links) while earning points, leveling up and collecting rewards. </summary> - [Close this issue](https://contributors.gitlab.com/manage-issue?action=close&projectId=278964&issueIid=631759) </details> <!--IssueSummary end--> ## Goal Goal of this issue is to assess the parity between validity checks and the default ruleset in [secret-detection-rules](https://gitlab.com/gitlab-org/security-products/secret-detection/secret-detection-rules). For every detection rule: - Either a validity check exists. - One can be built. - Or there is a recorded reason it cannot. ## Legend | Class | Sub-class | Meaning | Rules | | -------------- | --------- | ------- | ----- | | done | – | A validity check already exists for the rule. | 38 | | viable | – | A validity check can be generated for this rule. | 24 | | not-verifiable | – | The value is not verifiable, e.g. private keys, OAuth client IDs, webhooks, or a refresh token. | 36 | | needs-rework | – | A validity check is possible, but something must change first: | 119 | | | Rule change | The rule captures the wrong value, so the finding extract cannot be verified until the rule is fixed. | 8 | | | Region-split hosts | The endpoint depends on the customer's host region, so a live token can look dead on the wrong host. | 8 | | | Generator change | One credential and a fixed endpoint, but the request needs a generator feature that does not exist yet. | 16 | | | Code change | The check needs changes to the code, e.g. in `TokenLookupService` to lookup token across two models. | 6 | | | Hand-written client | The response is not expressible as a status-code mapping. | 3 | | | Parked | The HTTP status does not tell a live token from a dead one, so the check needs a hand-written client. | 5 | | | Multi-token validation | The detected value only authenticates together with a second component that is not detected. | 40 | | | Per-customer endpoint | The token only works against a host that cannot be derived from the detected secret. | 33 | ## Classifications Totals: Done (38), Viable (24), Needs Rework (119), Not Verifiable (36). Below are the classifications we have arrived at so far. ### Done (38) Partners and External Vendors (25) - `AWS` - `anthropic_key` - `DataDogAPIKey` - `GCP API key` - `GCP OAuth client secret` - `Google (GCP) Service-account` - `Github Personal Access Token` - `GithubFineGrainedPersonalAccessToken` - `Github OAuth Access Token` - `GithubAppInstallationToken` - `Heroku API Key` - `open ai token` - `OpenAiProjectKey` - `OpenAiServiceAccountKey` - `OpenAiServiceAdminKey` - `Postman API token` - `Sendgrid API token` - `StripeLiveSecretKey` - `StripeLiveRestrictedKey` - `StripeLivePublishableKey` - `StripeLiveShortSecretKey` - `StripeTestSecretKey` - `StripeTestRestrictedKey` - `StripeTestPublishableKey` - `StripeTestShortSecretKey` GitLab Tokens (13) - `gitlab_ci_build_token` - `gitlab_deploy_token` - `gitlab_feed_token_v2` - `gitlab_incoming_email_token` - `gitlab_kubernetes_agent_token` - `gitlab_oauth_app_secret` - `gitlab_personal_access_token` - `gitlab_personal_access_token_routable` - `gitlab_personal_access_token_routable_versioned` - `gitlab_pipeline_trigger_token` - `gitlab_runner_auth_token` - `gitlab_runner_auth_token_routable` - `gitlab_scim_oauth_token` ### Viable (24) | Rule | Endpoint | Invalid code (tested) | | ---- | -------- | --------------------- | | `npm access token`<br>!258832 DONE :white_check_mark: | `registry.npmjs.org/-/whoami` | 401 | | `AdafruitIOKey`<br>!260856 DONE :white_check_mark: | `io.adafruit.com/api/v2/user` | 401 | | `AsanaPersonalAccessTokenV1`<br>!260857 DONE :white_check_mark: | `app.asana.com/api/1.0/users/me` | 401 | | `AsanaPersonalAccessTokenV2`<br>!260857 DONE :white_check_mark: | `app.asana.com/api/1.0/users/me` | 401 | | `AtlassianApiKey` | `api.atlassian.com/admin/v1/orgs` | 401 | | `Beamer API token`<br>!260861 DONE :white_check_mark: | `api.getbeamer.com/v0/posts` | 401 | | `CDSCanadaNotifyAPIKey`<br>!260874 DONE :white_check_mark: | `api.notification.canada.ca/notifications` | 403 (send full match as `Authorization` header, i.e. `auth_style: header`) | | `ContentfulPersonalAccessToken` | `api.contentful.com/spaces` | 401 (confirm EU-residency orgs authenticate here) | | `digitalocean-access-token`<br>!260865 DONE :white_check_mark: | `api.digitalocean.com/v2/account` | 401 | | `digitalocean-pat`<br>!260865 DONE :white_check_mark: | `api.digitalocean.com/v2/account` | 401 | | `Doppler API token`<br>!260867 DONE :white_check_mark: | `api.doppler.com/v3/me` | 401 | | `Doppler Service token`<br>!260867 DONE :white_check_mark: | `api.doppler.com/v3/me` | 401 | | `Fastly API token`<br>!260870 DONE :white_check_mark: | `api.fastly.com/tokens/self` | 401 | | `Flutterwave secret key`<br>!260872 DONE :white_check_mark: | `api.flutterwave.com/v3/balances` | 401 | | `FlutterwaveProdSecretKey`<br>!260872 DONE :white_check_mark: | `api.flutterwave.com/v3/balances` | 401 | | `Frame.io API token` | `api.frame.io/v2/me` | 401 | | `Hubspot API token`<br>!260877 DONE :white_check_mark: | `api.hubapi.com/crm/v3/objects/contacts` | 401 | | `HuggingFaceUserAccessToken`<br>!260884 DONE :white_check_mark: | `huggingface.co/api/whoami-v2` | 401 | | `Ionic API token` | `api.ionicjs.com/users/self` | 401 | | `Lob API Key` | `api.lob.com/v1/addresses` (basic, key as username) | 401 | | `Rubygem API token` | `rubygems.org/api/v1/profile/me.json` (`api_key.yaml` from older docs is 410 Gone) | 401 | | `TailscaleApiAccessToken`<br>!260886 DONE :white_check_mark: | `api.tailscale.com/api/v2/tailnet/-/devices` (basic, key as username) | 401 | | `WakaTimeAPIKey` | `wakatime.com/api/v1/users/current` (basic, key as username) | – | | `Sendinblue API token` | `api.brevo.com/v3/account` | 401 | ## Needs Rework (119) ### Rule Change (8) > The detection rule needs fixing before any validity check makes sense. Endpoint notes apply after the fix. - `Duffel API token` – capture group reports only `test` or `live` as the secret; API also requires a fixed version header - `Slack token` – capture-group fix upstream (secret-detection-rules!219; needs [rule change policy](https://gitlab.com/gitlab-org/gitlab/-/work_items/629010) finalized first) - `CircleCI access tokens` – one rule spans `CCIPAT_` and `CCIPRJ_`, which validate against different endpoints - `Mailchimp API key` – capture hardcodes `-us20`; the endpoint also derives from the key's datacenter suffix - `Instagram access token` – trailing character included in the match - `Meta access token` – trailing character included in the match - `Oculus access token` – trailing character included in the match - `Github App Token` – capture-group fix upstream (secret-detection-rules!229; needs [rule change policy](https://gitlab.com/gitlab-org/gitlab/-/work_items/629010) finalized first) ### Region-split Hosts (8) > Separate US/EU (or AU) hosts and the finding does not encode its region. A live key tested on the wrong host returns 401/403 and would misreport as inactive. Needs a multi-endpoint client (hand-written today) or generator support for trying hosts in sequence. - `Intercom API token` - `LangChainAPIKey` (403 for bad credentials, tested) - `New Relic REST API Key` - `Onfido Live API Token` – region is in the prefix (`api_live.`, `api_live_us.`, `api_live_ca.`): - Note that auth prefix `Token token=` is already supported (!261893) - `PostHogPersonalAPIkey` - `Segment Public API token` (403 for bad credentials, tested) - `Typeform API token` (403 for bad credentials, tested) - `GrafanaCloudAccessPolicyToken` – the Cloud API requires a `region` query parameter the token does not encode ### Generator Changes (16) > The request needs something the generator cannot produce yet: an auth style, a header prefix, a request body or the token in the query string. `basic_password` auth style - !258812 :white_check_mark: - `AzurePersonalAccessToken` - `Mailgun private API token` POST request body: - `Dropbox short lived API token` - `Dropbox long lived API token` - `DropboxAppAccessToken` - `Linear API token` - `New Relic user API Key` (also region-split) - `HighnoteLiveSecretKey` - `HighnoteTestSecretKey` Auth header value prefix (`OAuth <t>`, `Token token=<t>`, `AccessKey <t>`...) - !261893 :white_check_mark: - `messagebird-api-token` !262235 DONE :white_check_mark: - `Pulumi API token` !261915 DONE :white_check_mark: - `Shippo API token` !262192 DONE :white_check_mark: - `Shippo Test API token` !262192 DONE :white_check_mark: - `Yandex.Cloud API Key` !263436 - in review :hourglass_flowing_sand: Query-param auth (deliberately unsupported today): - `Mapbox API token` - `MapboxSecretApiToken` ### Code Changes (6) > The rule needs no change, but our code must clean the detected value (quotes, a URL parameter) or look it up in more than one model before a validity check can run. - `EasyPost API token` !260411 DONE :white_check_mark: - `EasyPost test API token` !260420 DONE :white_check_mark: - `GoCardless API token` !261169 and !261172 DONE :white_check_mark: - `Hashicorp Terraform user/org API token` https://gitlab.com/gitlab-org/gitlab/-/merge_requests/259344 - DONE :white_check_mark: - `gitlab_feed_token` – the match includes `feed_token=` URL parameter; we need to strip it, then look up with `FEED_TOKEN_CONFIG` - `gitlab_runner_registration_token` – `TokenLookupService#find` reads one model per token type, but this token is stored on `Project` and `Group` ### Hand-written client (3) > The vendor returns the verdict in the response body, or the token can only be checked offline, so a status-code mapping cannot express the check. - `SlackAppLevelToken` – the API answers HTTP 200 with the verdict in the body - `SlackAppConfigurationToken` – same - `AzureEntraIDToken` – offline JWT validation; no unambiguous HTTP code ### Parked (5) > No HTTP status reliably means a dead token, or the token expires too soon to be worth checking, so we do not build a check for now. - `AWSBedrockKey` – tested 403 for bad auth, but AWS 403 also means valid-without-permission; regional endpoints - `AWSBedrockShortLivedKey` – same - `AdobeIMSAccessToken` – 24h lifetime; userinfo needs client context - `FigmaPersonalAccessToken` – scoped tokens: 403 covers both dead and live-but-wrong-scope (tested 403 for garbage) - `GCPVertexExpressModeKey` – query auth and no unambiguous dead code ### Multi-token Validation (40) > The detected value only authenticates together with a second value, such as a client ID or a username, that the finding does not contain. - `Twilio API Key` – API secret - `Twilio Account SID` – API secret - `DockerPersonalAccessToken` – username - `AzureEntraClientSecret` – client + tenant id - `AWSSecretAccessKey` – access key id - `AWSSessionToken` – key pair - `Atlassian API token` – email as basic username - `AtlassianUserApiToken` – email as basic username - `Adobe Client Secret` – client id - `Alibaba AccessKey ID` – pair - `Alibaba Secret Key` – pair - `Asana Client Secret` – client id - `Auth0ApplicationClientSecret` – domain + client id - `Bitbucket client secret` – client id - `Clojars API token` – username - `Contentful delivery API token` – space id - `Contentful preview API token` – space id - `Discord client secret` – client id - `Dropbox API secret/key` – app pair - `Facebook token` – debug_token needs app credentials - `Finicity API token` – partner credentials - `Finicity client secret` – partner id - `Intercom client secret/ID` – pair - `Linear client secret/ID` – pair - `Linkedin Client secret` – client id - `MaxMind License Key` – account id - `OktaClientSecret` – client id - `Planetscale API token` – token id - `PlanetscaleAppSecret` – app id - `PlanetscaleOAuthSecret` – client id - `PostmanCollectionAccessKey` – collection id - `Sendinblue SMTP token` – pairs with account login over SMTP - `Shopify shared secret` – app context - `ShopifyPartnerAPIToken` – organization id - `TailscaleOauthClientSecret` – client id - `TencentCloudSecretID` – pair - `Twitch API token` – client id (the rule targets client secrets, and `/oauth2/validate` returns 401 for a live one, tested) - `Twitter token` – app/OAuth context - `VolcengineAccessKeyID` – pair - `Yandex.Cloud AWS API compatible Access Secret` – key id ### Per-customer Endpoint (33) > The token only works against the customer's own host, such as a self-hosted instance or an org subdomain, and the finding does not contain that host. - `AivenServicePassword` - `ArtifactoryApiKey` - `ArtifactoryIdentityToken` - `JfrogPlatformAccessToken` - `Databricks API token` - `DynatracePlatformToken` - `GrafanaServiceAccountToken` - `Hashicorp Vault batch token` - `HashicorpVaultServiceToken` - `KubernetesServiceAccToken` - `OktaAPIToken` - `OktaAPITokenHeader` - `Planetscale password` - `Shopify access token` - `Shopify custom app access token` - `Shopify private app access token` - `SonarQubeUserToken` - `SonarQubeGlobalAnalysisToken` - `SonarQubeProjectAnalysisToken` - `SplunkAuthToken` - `SplunkHECToken` - `AzureAPIManagementDirectKey` - `AzureAPIManagementGatewayKey` - `AzureAppConfigConnectionString` - `AzureCommServicesConnectionString` - `AzureCosmosDBCredentials` - `AzureEventGridAccessKey` - `AzureEventGridSAS` - `AzureFunctionsAPIKeyViaHeader` - `AzureFunctionsAPIKeyViaURL` - `AzureLogicAppSAS` - `AzureOpenAIAPIKey` - `AzureSignalRAccessKey` ## Not Verifiable (36) - `SSH private key` - `SSH (DSA) private key` - `SSH (EC) private key` - `RSA private key` - `PGP private key` - `PKCS8 private key` - `Age secret key` - `Password in URL` - `PyPI upload token` – upload-only - `Mailgun public validation key` – validation API discontinued - `Mailgun webhook signing key` - `New Relic ingest browser API token` - `New Relic ingest browser API token v2` - `New Relic user API ID` – identifier - `Yandex.Cloud IAM Cookie v1 - 1` - `Yandex.Cloud IAM Cookie v1 - 2` - `digitalocean-refresh-token` – exchange-only, minting side effects - `SlackAppConfigurationRefreshToken` – exchange-only; checking it rotates the token pair - `Github Refresh Token` – exchange-only; checking it rotates the token pair - `Slack Webhook` – validating means posting - `Discord API key` – generic context hex, not a verifiable credential - `Lob Publishable API Key` - `Flutterwave public key` - `Flutterwave encrypted key` - `FlutterwaveProdPublicKey` - `FlutterwaveProdEncryptedKey` - `PostHogProjectAPIkey` – public by design - `AmazonOAuthClientID` – identifier - `Adobe Client ID (Oauth Web)` – identifier - `Discord client ID` – identifier - `Linkedin Client ID` – identifier - `Bitbucket client ID` – identifier - `Asana Client ID` – identifier - `AWSCognitoIdentityPoolID` – identifier - `MessageBird API client ID` – identifier - `TailscalePersonalAuthKey` – registers devices ## Suggested Order 1. Keep generating from the viable list. 2. Merge !258812 (`basic_password`) to unlock Azure DevOps PAT and Mailgun. 3. Fold the rule-change bucket into the rule-changes policy work – those 8 rules gate their verifiers. 4. Decide the region-split class once (generator multi-host support vs hand-written) – it gates 8 rules including several popular vendors. 5. Generator changes by unlock count: value prefix (5), POST body (7), query auth (2). 6. Multi-token validation design covers the 40-rule bucket; tracked separately.