Tune secret detection rules

Why are we doing this work

Some rules are noisier than others. The rules in the Secret Detection analyzer are not immune to this reality and could use some attention. As secret detection continues to be a more widely used feature category, we should take a moment to tune some of the rules which are more prone to flagging false positives.

Relevant links

• keyhacks project
  • This project enumerates methods by which keys reported from specific services can be validated. Might be an interesting means of validating detected secrets for true positives?
• GitHub Universe secret detection notes

Non-functional requirements

• Documentation:
• Feature flag:
• Performance:
• Testing:

Implementation plan

changed milestone to %Backlog

added Category:Secret Detection backend devopssecure groupstatic analysis sectionsec typemaintenance labels

added to epic &6435

added typefeature label

@zrice - in our last Static Analysis weekly, you mentioned a few rules which could use some tuning. Might you be able to populate some of those in this issue? We'd like to pull this work into the coming quarter while we're focused on reliability and availability work.

@twoodham Taking a look at the Secret Analyzer's gitleaks config it looks like the rule that I was concerned about being noisey, twitter, is not an issue with the Secret Analyzer. The twitter token rule is surrounded by quotations in the analyzer but not in the default gitleaks config (this has been fixed in v7.6.0)

That said, there is some work to be done with the rules:

1. Determine which secrets from the semgrep ruleset would be worthwhile to include. Porting the semgrep rules to gitleaks rules is simple since they are mostly just regex.
2. Add more restrictive allow-list regex/files. See https://github.com/zricethezav/gitleaks/issues/575#issuecomment-898476714 for additional context.
3. Add a tree-sitter false positive reduction feature to gitleaks.

Add more restrictive allow-list regex/files. See https://github.com/zricethezav/gitleaks/issues/575#issuecomment-898476714 for additional context.

This would be considered ~"feature::maintenance" so I will work on refining this issue around this point.

added [deprecated] Accepting merge requests label

changed milestone to %14.4

added workflowplanning breakdown label

marked this issue as related to #342539

This might be a simple fix related to this issue: #342539

It's getting late in %14.4 and we haven't started work on this issue. Moving to %14.5 as a result and would like to get this maintenance in during that release's window.

changed milestone to %14.5

removed [deprecated] Accepting merge requests label

changed milestone to %Backlog

removed workflowplanning breakdown label

added [deprecated] Accepting merge requests label

@zrice - might you have some rules in mind that should be targeted for tuning?

In Early 2022, GitHub will start doing commit rejection for "well-identifiable" secrets, here's an example of 5, if we can find a list of these, I'd like to get rules added:

GitHub also claimed they worked with all these companies to get them to swithc to well identifiable secrets, so we can probably look at their docs and find patterns:

Mentioned in the public GitHub session here: https://youtu.be/ABjrJMjbdRw?t=866

GitHub has kindly also provided a list of these API key slugs here: https://docs.github.com/en/code-security/secret-scanning/about-secret-scanning#about-secret-scanning-for-private-repositories

Provider	Supported secret	API slug
Adafruit IO	Adafruit IO Key	adafruit_io_key
Adobe	Adobe Device Token	adobe_device_token
Adobe	Adobe Service Token	adobe_service_token
Adobe	Adobe Short-Lived Access Token	adobe_short_lived_access_token
Adobe	Adobe JSON Web Token	adobe_jwt
Alibaba Cloud	Alibaba Cloud Access Key ID	alibaba_cloud_access_key_id
Alibaba Cloud	Alibaba Cloud Access Key Secret	alibaba_cloud_access_key_secret
Amazon Web Services (AWS)	Amazon AWS Access Key ID	aws_access_key_id
Amazon Web Services (AWS)	Amazon AWS Secret Access Key	aws_secret_access_key
Amazon Web Services (AWS)	Amazon AWS Session Token	aws_session_token
Amazon Web Services (AWS)	Amazon AWS Temporary Access Key ID	aws_temporary_access_key_id
Asana	Asana Personal Access Token	asana_personal_access_token
Atlassian	Atlassian API Token	atlassian_api_token
Atlassian	Atlassian JSON Web Token	atlassian_jwt
Atlassian	Bitbucket Server Personal Access Token	bitbucket_server_personal_access_token
Azure	Azure DevOps Personal Access Token	azure_devops_personal_access_token
Azure	Azure SAS Token	azure_sas_token
Azure	Azure Service Management Certificate	azure_management_certificate
Azure	Azure Storage Account Key	azure_storage_account_key
Beamer	Beamer API Key	beamer_api_key
Checkout.com	Checkout.com Production Secret Key	checkout_production_secret_key
Checkout.com	Checkout.com Test Secret Key	checkout_test_secret_key
Clojars	Clojars Deploy Token	clojars_deploy_token
CloudBees CodeShip	CloudBees CodeShip Credential	codeship_credential
Contentful	Contentful Personal Access Token	contentful_personal_access_token
Databricks	Databricks Access Token	databricks_access_token
Discord	Discord Bot Token	discord_bot_token
Doppler	Doppler Personal Token	doppler_personal_token
Doppler	Doppler Service Token	doppler_service_token
Doppler	Doppler CLI Token	doppler_cli_token
Doppler	Doppler SCIM Token	doppler_scim_token
Doppler	Doppler Audit Token	doppler_audit_token
Dropbox	Dropbox Access Token	dropbox_access_token
Dropbox	Dropbox Short Lived Access Token	dropbox_short_lived_access_token
Duffel	Duffel Live Access Token	duffel_live_access_token
Duffel	Duffel Test Access Token	duffel_test_access_token
Dynatrace	Dynatrace Access Token	dynatrace_access_token
Dynatrace	Dynatrace Internal Token	dynatrace_internal_token
EasyPost	EasyPost Production API Key	easypost_production_api_key
EasyPost	EasyPost Test API Key	easypost_test_api_key
Facebook	Facebook Access Token	facebook_access_token
Fastly	Fastly API Token	fastly_api_token
Finicity	Finicity App Key	finicity_app_key
Flutterwave	Flutterwave Live API Secret Key	flutterwave_live_api_secret_key
Flutterwave	Flutterwave Test API Secret Key	flutterwave_test_api_secret_key
Frame.io	Frame.io JSON Web Token	frameio_jwt
Frame.io	Frame.io Developer Token	frameio_developer_token
FullStory	FullStory API Key	fullstory_api_key
GitHub	GitHub Personal Access Token	github_personal_access_token
GitHub	GitHub OAuth Access Token	github_oauth_access_token
GitHub	GitHub Refresh Token	github_refresh_token
GitHub	GitHub App Installation Access Token	github_app_installation_access_token
GitHub	GitHub SSH Private Key	github_ssh_private_key
GoCardless	GoCardless Live Access Token	gocardless_live_access_token
GoCardless	GoCardless Sandbox Access Token	gocardless_sandbox_access_token
Google	Firebase Cloud Messaging Server Key	firebase_cloud_messaging_server_key
Google	Google API Key	google_api_key
Google	Google Cloud Private Key ID	google_cloud_private_key_id
Google	Google Cloud Storage Access Key Secret	google_cloud_storage_access_key_secret
Google	Google Cloud Storage Service Account Access Key ID	google_cloud_storage_service_account_access_key_id
Google	Google Cloud Storage User Access Key ID	google_cloud_storage_user_access_key_id
Google	Google OAuth Client ID	google_oauth_client_id
Google	Google OAuth Client Secret	google_oauth_client_secret
Grafana	Grafana API Key	grafana_api_key
Hashicorp Terraform	Terraform Cloud / Enterprise API Token	terraform_api_token
Hubspot	Hubspot API Key	hubspot_api_key
Intercom	Intercom Access Token	intercom_access_token
Ionic	Ionic Personal Access Token	ionic_personal_access_token
Ionic	Ionic Refresh Token	ionic_refresh_token
JFrog	JFrog Platform Access Token	jfrog_platform_access_token
JFrog	JFrog Platform API Key	jfrog_platform_api_key
Linear	Linear API Key	linear_api_key
Linear	Linear OAuth Access Token	linear_oauth_access_token
Lob	Lob Live API Key	lob_live_api_key
Lob	Lob Test API Key	lob_test_api_key
Mailchimp	Mailchimp API Key	mailchimp_api_key
Mailgun	Mailgun API Key	mailgun_api_key
MessageBird	MessageBird API Key	messagebird_api_key
New Relic	New Relic Personal API Key	new_relic_personal_api_key
New Relic	New Relic REST API Key	new_relic_rest_api_key
New Relic	New Relic Insights Query Key	new_relic_insights_query_key
New Relic	New Relic License Key	new_relic_license_key
npm	npm Access Token	npm_access_token
NuGet	NuGet API Key	nuget_api_key
Onfido	Onfido Live API Token	onfido_live_api_token
Onfido	Onfido Sandbox API Token	onfido_sandbox_api_token
OpenAI	OpenAI API Key	openai_api_key
Palantir	Palantir JSON Web Token	palantir_jwt
PlanetScale	PlanetScale Database Password	planetscale_database_password
PlanetScale	PlanetScale OAuth Token	planetscale_oauth_token
PlanetScale	PlanetScale Service Token	planetscale_service_token
Plivo	Plivo Auth ID	plivo_auth_id
Plivo	Plivo Auth Token	plivo_auth_token
Postman	Postman API Key	postman_api_key
Proctorio	Proctorio Consumer Key	proctorio_consumer_key
Proctorio	Proctorio Linkage Key	proctorio_linkage_key
Proctorio	Proctorio Registration Key	proctorio_registration_key
Proctorio	Proctorio Secret Key	proctorio_secret_key
Pulumi	Pulumi Access Token	pulumi_access_token
PyPI	PyPI API Token	pypi_api_token
RubyGems	RubyGems API Key	rubygems_api_key
Samsara	Samsara API Token	samsara_api_token
Samsara	Samsara OAuth Access Token	samsara_oauth_access_token
SendGrid	SendGrid API Key	sendgrid_api_key
Sendinblue	Sendinblue API Key	sendinblue_api_key
Sendinblue	Sendinblue SMTP Key	sendinblue_smtp_key
Shippo	Shippo Live API Token	shippo_live_api_token
Shippo	Shippo Test API Token	shippo_test_api_token
Shopify	Shopify App Shared Secret	shopify_app_shared_secret
Shopify	Shopify Access Token	shopify_access_token
Shopify	Shopify Custom App Access Token	shopify_custom_app_access_token
Shopify	Shopify Private App Password	shopify_private_app_password
Slack	Slack API Token	slack_api_token
Slack	Slack Incoming Webhook URL	slack_incoming_webhook_url
Slack	Slack Workflow Webhook URL	slack_workflow_webhook_url
SSLMate	SSLMate API Key	sslmate_api_key
SSLMate	SSLMate Cluster Secret	sslmate_cluster_secret
Stripe	Stripe API Key	stripe_api_key
Stripe	Stripe Live API Secret Key	stripe_live_secret_key
Stripe	Stripe Test API Secret Key	stripe_test_secret_key
Stripe	Stripe Live API Restricted Key	stripe_live_restricted_key
Stripe	Stripe Test API Restricted Key	stripe_test_restricted_key
Stripe	Stripe Webhook Signing Secret	stripe_webhook_signing_secret
Tableau	Tableau Personal Access Token	tableau_personal_access_token
Telegram	Telegram Bot Token	telegram_bot_token
Tencent Cloud	Tencent Cloud Secret ID	tencent_cloud_secret_id
Twilio	Twilio Access Token	twilio_access_token
Twilio	Twilio Account String Identifier	twilio_account_sid
Twilio	Twilio API Key	twilio_api_key

@tmccaslin

GitHub has kindly also provided a list of these API key slugs here: https://docs.github.com/en/code-security/secret-scanning/about-secret-scanning#about-secret-scanning-for-private-repositories

Slightly confused here, is this api key slugs for their revocation api or slugs from the api key provider?

Yeah you're right, these are just internal identifiers rather than slug prefixes

Stripe for example is:

• pk_test
• sk_test
• pk_live
• sk_live

We just need to go through vendor API docs to see if they have API slugs that are easy to detect.

I'm happy to help do this work next week. Maybe we just have a fun 30 minute meeting in a google doc for anyone to join and go collect key patterns.

@tmccaslin I'm starting to work my way through this list and compile some regexes. This involves creating a bunch of accounts for these providers.

Made it down to MessageBird! The list is progressing

According to my spreadsheet, we can add 47 rules with unique prefixes/patterns that make them easily identifiable!

removed typefeature label